CodeT5 input for APPS/MBPP problems #27

Activity

1. henryhungle commented on Feb 23, 2023

   

   henryhungle

   on Feb 23, 2023

   Collaborator

   More actions

   We followed the default formats that are used in these benchmarks. For APPS, the format is defined here in the code:

   CodeRL/datasets/apps_dataset.py

   Line 308 in 51db4ff

   q_str = "\nQUESTION:\n" + q_str + "\n" + s_str + "\n" + answer_type + "\nANSWER:\n"

   where q_str is the question description, s_str is the starter code (if any), answer_type is the type of problems in APPS (e.g. Call-based/ Standard-input).

   For MBPP, please refer to the original paper for the input format.
2. invade-art commented on Jul 5, 2024

   

   invade-art

   on Jul 5, 2024

   More actions

   We followed the default formats that are used in these benchmarks. For APPS, the format is defined here in the code:

   CodeRL/datasets/apps_dataset.py

   Line 308 in 51db4ff

   q_str = "\nQUESTION:\n" + q_str + "\n" + s_str + "\n" + answer_type + "\nANSWER:\n"

   where q_str is the question description, s_str is the starter code (if any), answer_type is the type of problems in APPS (e.g. Call-based/ Standard-input).

   For MBPP, please refer to the original paper for the input format.

   hi,where is the original paper (the MBPP).I want to test model in the mbpp,but i dont konw how to got it