Context
This doc explains how to use the Prolific API to create a study with AI Taskers, a group of qualified participants who excel at model evaluation, testing, and alignment. To learn more about how we qualify AI Taskers, check out this article.
We’ve added a new filter (ai-taskers) which allows you to select AI Taskers as participants for your study. We recommend using them for model evaluation and benchmarking, side-by-side performance comparisons, factuality and accuracy assessments, and generative tasks.
Changes to GET filters api
A researcher can get the list of filters by making a GET filters request as specified in our API doc
The response will now include a new filter:
{
"filter_id": "ai-taskers",
"title": "AI taskers",
"description": "",
"question": "AI tasker?",
"researcher_help_text": "",
"participant_help_text": "",
"category": "AI taskers",
"tags": [],
"type": "select",
"data_type": "ChoiceID",
"choices": {
"0": "Qualified AI taskers"
}
}Create a study for our AI Taskers
Using the response above, you can create a study with a new filter id “ai-task-experts” following our existing create study API doc
You’ll need to update the filters attribute in the request body of the create study API as such:
{
"internal_name": "test",
"name": "khoov test",
"description": "test",
"external_study_url": "<<https://www.test.com>",
"reward": 15,
"total_available_places": 100,
"publish_at": null,
"prolific_id_option": "not_required",
"audience": "standard_sample",
"device_compatibility": [
"mobile",
"desktop",
"tablet"
],
"peripheral_requirements": [],
"url_prefix": "<https://test.prolific.com/submissions/complete?cc=>",
"selected_ai_tasker_group": "0",
"location": true,
"estimated_completion_time": 1,
"maximum_allowed_time": 13,
"content_warnings": [],
"study_labels": [],
"pii": {
"enabled": false
},
"is_custom_screening": false,
"filter_set_id": null,
"filter_set_version": null,
"has_credentials": false,
**"filters": [
{
"filter_id": "ai-taskers",
"selected_values": [
"0"
]
}
],**
"study_type": "SINGLE",
"completion_codes": [
{
"code": "C1FF2ZY9",
"code_type": "COMPLETED",
"actions": [
{
"action": "MANUALLY_REVIEW"
}
]
}
],
"submissions_config": {
"max_submissions_per_participant": 1,
"max_concurrent_submissions": -1
}
}Please note that the selected_values in the filters attribute is “0” which match General AI Taskers choices from the get filters response above.
The response will be 200 OK with an example payload of:
{
"id": "60d9aadeb86739de712faee0",
"name": "Study about API's",
"internal_name": "WIT-2021 Study about API's version 2",
"description": "This study aims to determine how to make a good public API",
"external_study_url": "<https://eggs-experriment.com?participant=#{{%PROLIFIC_PID%}>}",
"prolific_id_option": "url_parameters",
"completion_codes": [
{
"code": "ABC123",
"code_type": "COMPLETED",
"actions": [
{
"action": "AUTOMATICALLY_APPROVE"
}
]
},
{
"code": "DEF234",
"code_type": "FOLLOW_UP_STUDY",
"actions": [
{
"action": "AUTOMATICALLY_APPROVE"
},
{
"action": "ADD_TO_PARTICIPANT_GROUP",
"participant_group": "619e049f7648a4e1f8f3645b"
}
]
}
],
"total_available_places": 30,
"estimated_completion_time": 5,
"maximum_allowed_time": 25,
"reward": 100,
"device_compatibility": [
"desktop"
],
"peripheral_requirements": [],
**"filters": [
{
"filter_id": "ai-taskers",
"selected_values": [
"0"
]
}
],**
"filter_set_id": null,
"filter_set_version": null,
"status": "UNPUBLISHED",
"study_labels": [
"interview"
],
"content_warnings": [
"sensitive"
],
"content_warning_details": "Experiences with hateful activities, experiences with self-injury and harmful behaviour",
"is_custom_screening": false
}