Skip to content

Generating max_num_tokens.csv for Different Hardware Environments #30

Open

Description

@village-way

In the part of the experimental model for finding the optimal placement strategy, there are three files in the simdistserve/estimators/profile_data directory, one of which is named max_num_tokens.csv. I am very curious about how this file is generated in different environments, because the other two JSON files can be generated based on the code in evaluation/0-test-single-forward-performance. However, I have not found a method for generating this CSV file. Based on my understanding, this file should be associated with different hardware or GPU environments, and the profiles in different environments should be different. I am eagerly awaiting your response. Thank you very much!

image

Activity

  1. lzc-code commented on Oct 17, 2024

    @lzc-code

    I met the same problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions