RAMP API

This is the full API documentation of the different RAMP packages.

RAMP database

ramp_database.model: the database model

The ramp_database.model defines the database structure which is used for the RAMP events.

The database schema is summarized in the figure below:

General tables

model.Extension(**kwargs)	Extension table.
model.Keyword(**kwargs)	Keyword table.
model.UserInteraction([interaction, user, …])	UserInteraction table.

User-related tables

model.Team(name, admin[, is_individual])	Team table.
model.User(name, hashed_password, lastname, …)	User table.

Event-related tables

model.CVFold(**kwargs)	CVFold table.
model.Event(problem_name, name, event_title, …)	Event table.
model.Problem(name, path_ramp_kit, …[, …])	Problem table.
model.ScoreType(name, is_lower_the_better, …)	ScoreType table.
model.Workflow(name)	Workflow table.

Submission-related tables

model.HistoricalContributivity(**kwargs)	HistoricalContributivity table.
model.Submission(name, event_team[, …])	Submission table.
model.SubmissionScore(**kwargs)	SubmissionScore table.
model.SubmissionFile(**kwargs)	SubmissionFile table.
model.SubmissionFileType(**kwargs)	SubmissionFileType table.
model.SubmissionOnCVFold(submission, cv_fold)	SubmissionOnCVFold.
model.SubmissionScoreOnCVFold(**kwargs)	SubmissionScoreOnCVFold table.
model.SubmissionSimilarity(**kwargs)	SubmissionSimilarity table.

Relationship tables

model.EventTeam(event, team)	EventTeam table.
model.EventAdmin(**kwargs)	EventAdmin table.
model.EventScoreType(event, score_type_object)	EventScoreType table.
model.ProblemKeyword(**kwargs)	ProblemKeyword table.
model.SubmissionFileTypeExtension(**kwargs)	SubmissionFileTypeExtension table.
model.WorkflowElement(workflow, …[, …])	WorkflowElement table.
model.WorkflowElementType(**kwargs)	WorkflowElementType table.

ramp_database.tools: the tools to communicate with the database

The ramp_database.tools provides some routine to get and set information from and in the database.

User-related database tools

Functions to act on an entry in the database

tools.team.ask_sign_up_team(session, …)	Register a team to a RAMP event without approving.
tools.user.approve_user(session, name)	Approve a user once it is created.
tools.user.make_user_admin(session, name)	Make a user a RAMP admin.
tools.user.set_user_access_level(session, name)	Change the access level of a user.
tools.team.sign_up_team(session, event_name, …)	Register a team to a RAMP event and submit the starting kit.

Functions to add new entries in the database

tools.user.add_user(session, name, password, …)	Add a new user in the database.
tools.user.add_user_interaction(session[, …])	Add a user interaction in the database.

Functions to get entries from the database

tools.team.get_event_team_by_name(session, …)	Get the event/team given an event and a user.
tools.user.get_user_by_name(session, name)	Get a user by his/her name.
tools.user.get_team_by_name(session, name)	Get a team by its name.
tools.user.get_user_interactions_by_name(session)	Get the user interactions.

Functions to set an entry in the database

tools.user.set_user_by_instance(session, …)

Set the information of a user.

Functions to delete an entry in the database

tools.user.delete_user(session, name)	Delete a user from the database.
tools.team.delete_event_team(session, …)	Delete a team from an RAMP event.

Event-related database tools

Functions to act on an entry in the database

tools.leaderboard.update_leaderboards(…[, …])	Update the leaderboards for a given event.
tools.leaderboard.update_user_leaderboards(…)	Update the of a user leaderboards for a given event.
tools.leaderboard.update_all_user_leaderboards(…)	Update the leaderboards for all users for a given event.

Functions to add new entries in the database

tools.event.add_event(session, problem_name, …)	Add a RAMP event in the database.
tools.event.add_event_admin(session, …)	Add an administrator event.
tools.event.add_keyword(session, name, …)	Add a keyword to the database.
tools.event.add_problem(session, …[, force])	Add a RAMP problem to the database.
tools.event.add_problem_keyword(session, …)	Add relationship between a keyword and a problem.
tools.event.add_workflow(session, …)	Add a new workflow.

Functions to delete entries in the database

tools.event.delete_event(session, event_name)	Delete an event from the database.
tools.event.delete_problem(session, problem_name)	Delete a problem from the database.
tools.event.delete_submission_similarity(…)	Delete the submission similarity associated with a submission.

Functions to get entries from the database

tools.event.get_event(session, event_name)	Get event from the database.
tools.event.get_event_admin(session, …)	Get an administrator event.
tools.event.get_keyword_by_name(session, name)	Get the keyword filtering by there name
tools.leaderboard.get_leaderboard(session, …)	Get a leaderboard.
tools.event.get_problem(session, problem_name)	Get problem from the database.
tools.event.get_problem_keyword_by_name(…)	Get a problem-keyword relationship given their names.
tools.event.get_workflow(session, workflow_name)	Get workflow from the database.

Submission-related database tools

Functions to act on an entry in the database

tools.submission.submit_starting_kits(…)

Submit all starting kits for a given event.

Functions to add new entries in the database

tools.database.add_extension(session, name)	Adding a new extension, e.g., ‘py’.
tools.submission.add_submission(session, …)	Create a submission in the database and returns an handle.
tools.database.add_submission_file_type(…)	Add a new submission file type, e.g., (‘code’, True, 10 ** 5).
tools.database.add_submission_file_type_extension(…)	Adding a new submission file type extension, e.g., (‘code’, ‘py’).
tools.submission.add_submission_similarity(…)	Add submission similarity entry.

Functions to get entries from the database

tools.submission.get_event_nb_folds(session, …)	Get the number of fold for a given event.
tools.database.get_extension(session, …)	Get extension from the database.
tools.submission.get_scores(session, …)	Get the scores for each fold of a submission.
tools.submission.get_source_submissions(…)	Get the submissions with which a user interacted.
tools.submission.get_submissions(session, …)	Get information about submissions from an event with a specific state optionally.
tools.submission.get_submission_by_id(…)	Get a submission given its id.
tools.submission.get_submission_by_name(…)	Get a single submission filtering by event, team, and submission names.
tools.submission.get_submission_error_msg(…)	Get the error message after that a submission failed to be processed.
tools.database.get_submission_file_type(…)	Get submission file type from the database.
tools.database.get_submission_file_type_extension(…)	Get submission file type extension from the database.
tools.submission.get_submission_max_ram(…)	Get the max amount RAM used by a submission during processing.
tools.submission.get_submission_state(…)	Get the state of a submission given its id.

Functions to set an entry in the database

tools.submission.set_bagged_scores(session, …)	Set the bagged scores in the database.
tools.submission.set_scores(session, …)	Set the scores in the database.
tools.submission.set_submission_error_msg(…)	Set the error message after that a submission failed to be processed.
tools.submission.set_submission_max_ram(…)	Set the max amount RAM used by a submission during processing.
tools.submission.set_submission_state(…)	Set the set of a submission.
tools.submission.set_time(session, …)	Set the timing information in the database.

Frontend-related database tools

tools.frontend.is_admin(session, event_name, …)	Whether or not a user is administrator or administrate an event.
tools.frontend.is_accessible_event(session, …)	Whether or not an event is public or and a user is registered to RAMP or and admin.
tools.frontend.is_accessible_leaderboard(…)	Whether or not a leaderboard is public or and a user is registered to RAMP or and admin.
tools.frontend.is_accessible_code(session, …)	Whether or not the user can look at the code submission.
tools.frontend.is_user_signed_up(session, …)	Whether or not user signed up to an event.
tools.frontend.is_user_sign_up_requested(…)	Whether or not user signed up to an event.

ramp_database.exceptions: type of errors raise by the database

The ramp_database.exceptions module include all custom warnings and errors used in ramp-database.

exceptions.DuplicateSubmissionError	Error to raise when a submission is already present in the database.
exceptions.MergeTeamError	Error to raise when the merging of teams is failing.
exceptions.MissingSubmissionFileError	Error to raise when the file submitted is not present in the supposed location.
exceptions.MissingExtensionError	Error to raise when the extension is not registered in the database.
exceptions.NameClashError	Error to raise when there is a duplicate in submission name.
exceptions.TooEarlySubmissionError	Error to raise when a submission was submitted to early.
exceptions.UnknownStateError	Error to raise when the state of the submission is unknown.

ramp_database.testing: functionalities to test database model and tools

The ramp_database.testing module create facility functions to test the tools and model of ramp-database.

testing.add_events(session)	Add events in the database.
testing.add_users(session)	Add dummy users in the database.
testing.add_problems(session)	Add dummy problems into the database.
testing.create_test_db(database_config, …)	Create an empty test database and the setup the files for RAMP.
testing.create_toy_db(database_config, …)	Create a toy dataset with couple of users, problems, events.
testing.setup_toy_db(session)	Only setup the database by adding some data.
testing.setup_ramp_kit_ramp_data(…[, …])	Clone ramp-kit and ramp-data repository and setup it up.
testing.setup_files_extension_type(session)	Setup the files’ extensions and types.
testing.sign_up_teams_to_events(session)	Sign up user to the events in the database.
testing.submit_all_starting_kits(session)	Submit all starting kits.

ramp_database.utils: setup and connect RAMP database

The ramp_database.utils module provides tools to setup and connect to the RAMP database.

utils.check_password(password, hashed_password)	Check if a password is the same than the hashed password.
utils.hash_password(password)	Hash a password.
utils.setup_db(config)	Create a sqlalchemy engine and session to interact with the database.
utils.session_scope(config)	Connect to a database and provide a session to make some operation.

RAMP engine

The RAMP engine is made of a dispatcher to orchestrate the training and evaluation of submissions which are processed using workers. Since there is a dispatcher for each RAMP event, we provide a daemon which will start dispatcher for each open event.

RAMP Dispatcher

daemon.Daemon(config, events_dir)	RAMP daemon starting dispatchers for open challenges.
dispatcher.Dispatcher(config, event_config)	Dispatcher which schedule workers and communicate with the database.

RAMP Workers

base.BaseWorker(config, submission)	Metaclass used to build a RAMP worker.
local.CondaEnvWorker(config, submission)	Local worker which uses conda environment to dispatch submission.
remote.DaskWorker(config, submission)	Dask distributed worker
aws.AWSWorker(config, submission)	Run RAMP submissions on Amazon.

RAMP frontend

The RAMP frontend is the development of the website. It uses extensively Flask.

create_app(config)

Create the RAMP Flask app and register the views.

ramp_frontend.views: views used on the frontend to interact with the user

The ramp_frontend.views defines all operations a user can need to deal with the RAMP backend.

General views

general.index()	Default landing page.
general.ramp()	RAMP description request.
general.teaching()	Page related to RAMP offers for teaching classes.
general.data_science_themes()	Page reviewing problems organized by ML themes.
general.keywords(keyword_name)	Page which give details about a keyword.

Authentication views

Admin views

admin.approve_users()	Approve new user to log-in and sign-up to events.
admin.approve_single_user(user_name)	Approve a single user.
admin.approve_sign_up_for_event(event_name, …)	Approve a user for a specific event.
admin.update_event(event_name)	Update the parameters of an event.
admin.user_interactions()	Show the user interactions recorded on the website.
admin.dashboard_submissions(event_name)	Show information about all submissions for a given event.

Leaderboard views

leaderboard.my_submissions(event_name)	Landing page of all user’s submission information.
leaderboard.leaderboard(event_name)	Landing page showing all user’s submissions information.
leaderboard.competition_leaderboard(event_name)	Landing page for the competition leaderboard for all users.
leaderboard.private_leaderboard(event_name)	Landing page for the private leaderboard.
leaderboard.private_competition_leaderboard(…)	Landing page for the private competition leaderboard.

Submission views

ramp.problems()	Landing page showing all the RAMP problems.
ramp.problem(problem_name)	Landing page for a single RAMP problem.
ramp.user_event(event_name)	Landing page for a given event.
ramp.sign_up_for_event(event_name)	Landing page to sign-up to a specific RAMP event.
ramp.sandbox(event_name)	Landing page for the user’s sandbox.
ramp.ask_for_event(problem_name)
ramp.credit(submission_hash)	The landing page to credit other submission when a user submit is own.
ramp.event_plots(event_name)	Landing page of the plot illustrating the score evolution over time for a specific RAMP event.
ramp.view_model(submission_hash, f_name)	Rendering submission codes using templates/submission.html.
ramp.view_submission_error(submission_hash)	Rendering submission codes using templates/submission.html.

Utilities

redirect.redirect_to_credit(submission_hash, …)	Redirect the page to the credit landing page.
redirect.redirect_to_sandbox(event, message_str)	Redirect the page to the sandbox landing page.
redirect.redirect_to_user(message_str[, …])	Redirect the page to the problem landing page.
visualization.score_plot(session, event)

ramp_frontend.forms: forms used in the website

The ramp_frontend.forms module defines the different forms used on the website.

forms.LoginForm(*args, **kwargs)	Login-in form.
forms.UserUpdateProfileForm(*args, **kwargs)	User profile form.
forms.UserCreateProfileForm(*args, **kwargs)	User profile form.
forms.CodeForm(*args, **kwargs)	Code form.
forms.SubmitForm(*args, **kwargs)	Submission name form.
forms.UploadForm(*args, **kwargs)	Submission uploading form.
forms.EventUpdateProfileForm(*args, **kwargs)	Form to update the parameters of an event.
forms.MultiCheckboxField(*args, **kwargs)	A form containing multiple checkboxes.
forms.ImportForm(*args, **kwargs)	The form allowing to select which model to view.
forms.CreditForm(*args, **kwargs)	Credit form.
forms.AskForEventForm(*args, **kwargs)	Form to ask for a new event.

ramp_frontend.testing: functionalities to test the frontend

The ramp_frontend.testing module contains all functions used to easily test the frontend.

testing.login(client, username, password)	Simulate a log-in from a user.
testing.logout(client)	Simulate a log-out.
testing.login_scope(client, username, password)	Context manager to log-in during the with scope.

ramp_frontend.utils: Utilities to ease sending email

The ramp_frontend.utils provides utilities to ease sending email.

utils.body_formatter_user(user)	Create the body of an email using the user information.
utils.send_mail(to, subject, body)	Send email using Flask Mail.

RAMP utils

The RAMP utils package is made of utilities used for managing configuration, deploying RAMP events, and other utilities shared between the different RAMP packages.

Configuration utilities

generate_flask_config(config)	Generate the configuration to deal with Flask.
generate_ramp_config(event_config[, …])	Generate the configuration to deploy RAMP.
generate_worker_config(event_config[, …])	Generate the configuration for RAMP worker from a configuration file.
read_config(config_file[, filter_section, …])	Read and parse the configuration file for RAMP.
testing.database_config_template()	Return the path a template database configuration file.
testing.ramp_config_template()	Return the path a template RAMP configuration file.

Datasets utilities

datasets.fetch_from_osf(path_data, metadata)

Fetch files from OSF storage.

datasets.OSFRemoteMetaData(filename, id, …)

Deployment utilities

deploy.deploy_ramp_event(config, event_config)

Deploy a RAMP event using a configuration file.

RAMP shared utilities

import_module_from_source(source, name)

Load a module from a Python source file.