PERCOM 2022 will provide the opportunity for authors of accepted papers to participate in an Artifact Evaluation (AE) process. Authors who choose to participate can provide supplementary materials as artifacts (tools, applications, data sets, etc.) that will be reviewed solely for functional completeness (e.g., does provided code compile and run as expected; do experimental results match expected results, etc.). Participating authors will be allotted two additional pages in a separate Artifacts section of the proceedings to describe the nature and contents of the artifact package. Submission of artifacts packages will have no bearing on paper acceptance or publication. However, papers that have verified artifacts or data will be associated with badges in the conference program and proceedings. Two badges will be available: Artifact-Certified and Result-Certified. Upon submission, participating authors should indicate which badge(s) the committee should consider for the submission.
Artifacts of interest include (but are not limited to) the following:
• Software, including implementations or simulations of systems or algorithms.
• Data repositories, including, e.g., logging data, system traces, survey raw data.
• Frameworks, including tools and services illustrating new approaches to pervasive computing systems and application development.
This list is not exhaustive; authors with questions about the suitability of an artifact should reach out to the chairs.
Artifact Submission
The submission process is straightforward and follows the following steps:
• Package software and datasets. Use a freely available tool, such as VirtualBox; Docker container image; git/mercurial/svn repositories; or zip/tar files.
• Create an artifact guide. The guide can be up to 2 pages and should be prepended to a preliminary version of the accepted paper (as a single PDF file). The guide’s contents will vary based on the artifact but it should describe minimal software and hardware requirements, how to obtain and deploy the artifact, and what a reviewer should expect (e.g., the expected experimental results, or steps to navigate an application). The artifact guide should include a link to a publicly downloadable artifact, whatever that means for your particular submission.
• Submit the artifact guide via EDAS at https://edas.info/newPaper.php?c=28747
• Commit to maintaining the artifact for at least three years. As part of the submission process, the authors will be asked to certify that they will maintain the artifact as described in the two-page description at the publicly accessible location indicated in the description for at least three years past the date of the conference (i.e., through March 2024).
The Review Process
The review process will have three phases:
• Initial Review. The reviewers will follow the instructions given in the artifact’s guide to ensure that artifacts can be deployed and run as described. Any bugs or questions will be documented as feedback.
• Artifact Revision. During the second phase, authors will be able to correct any bugs in their artifacts/guides to make sure reviewers can un-package and run the artifact before the deeper evaluation.
• Final evaluation. Reviewers perform their final evaluation to ensure that each artifact runs as expected and/or contains the expected data.
Note that the first two phases may be intertwined (several rounds) if authors and reviewers are quick their communication; every submission is guaranteed at least one opportunity for revision.
Best Artifact Award
A PERCOM 2022 Best Artifact Award will be presented during the conference.
Artifact Review Committee
Co-Chairs
- Christine Julien, University of Texas at Austin, USA
- Janick Edinger, University of Hamburg, Germany
Reviewers
TBA
Important Dates
- Registration of intent to submit an artifact: January 15, 2022
- Initial artifact submission deadline: January 22, 2022
- Artifact feedback and revision period: January 22-February 12, 2022
- Artifact evaluation notification (badge allotment): February 12, 2022 [tentative]
- Artifact guide camera ready deadline: February 15, 2022 [tentative]
For more important dates see here.
For further information about the artifact submission and evaluation, please contact the Artifacts track chair, Janick Edinger ([email protected])