Guidelines for Packaging AEC Submissions |
Below we outline guidelines on how to package artifacts for submission. We are open to possibilities that we have not considered below; however, in cases where systems ought to fit these options, we suggest authors use them.
In all cases, the artifacts should be accompanied by suitable documentation, to save committee members the burden of reverse-engineering what the authors intended (e.g., a dataset is useless without some explanation on how to browse the data; a tool without a quick tutorial will be very hard to use). In particular, please make concrete what claims you are making of the artifact, if these differ from the expectations set up by the paper. This is a place where you can tell us about difficulties we might encounter in using the artifact, or its maturity relative to the content of the paper. We are still going to evaluate the artifact relative to the paper, but this helps set expectations up front, especially in cases that might frustrate the reviewers without prior notice.
Create a submission with the same title as the paper. In the abstract field, include two things:
At this URL, give us access to:
If one of the authors is an AEC chair, you may not submit an artifact. You can, however, indicate in your paper that you were unable to submit an artifact due to the conflict-of-interest.
If you have a conflict-of-interest with anyone on the committee (including the three chairs), please indicate this in both the Web page and in your submission email.
Irrespective of the nature of the artifacts, authors should create a single Web page (whether on their site or a third-party file upload service) that contains the artifact, the paper, and all necessary instructions.
For artifacts where this would be appropriate, it would be helpful to also provide a self-contained bundle (including instructions) as a single file (.tgz or .zip; please avoid exotic compressors) for convenient offline use: imagine the reviewer who wants to download a single file to expand and work with during a train or bus commute.
If it would be helfpul, please feel free to include a video that demonstrates the artifact running or explaining how it should be run.
The artifact submission thus consists of just the URL and any credentials required to access the files.
We ask that, during the evaluation period, you not embed any analytics or other tracking in the Web site for the artifact or, if you cannot control this, that you not access this data. This is important for maintaining the confidentiality of reviewers. If for some reason you cannot comply with this, please notify the chairs immediately.
Authors should strongly consider one of the following methods to package the software components of their artifacts (though the AEC is open to other reasonable formats as well):
In all cases, authors should make a genuine effort to not learn the identity of the reviewers. This may mean turning off “call home” features or analytics, or only using systems with high enough usage that AEC accesses will not stand out. In all cases where tracing is unavoidable, the authors should warn the reviewers in advance so the reviewers can try to take adequate safeguards.
Non-code artifacts should preferably be in open document formats. For documents and reports, we invite the authors to use PDF, ODF, or RTF. We invite the authors to submit experimental data and results in CSV (preferred option), JSON, or XML file formats. In the special case that authors have to submit non-standard data formats, they should also provide suitable readers.