The ACM CHI Conference on Human Factors in Computing Systems is the premier international conference of Human-Computer Interaction (HCI). CHI – pronounced 'kai' – annually brings together researchers and practitioners from all over the world and from diverse cultures, backgrounds, and positionalities, who have as an overarching goal to make the world a better place with interactive digital technologies.

Categories
Uncategorized

Moving to a Revise and Resubmit Review Process for CHI 2022

Moving to a Revise and Resubmit Review Process for CHI 2022

Published: May 26, 2021 at 23:29 UTC

As an effort to improve paper quality, increase equity, and reflow the reviewing process, CHI 2022 Papers Chairs, Technical Program Chairs, and General Chairs have replaced the paper rebuttal mechanism with a Revise and Resubmit (R&R) reviewing process. R&R is a two-round reviewing process. At the end of a first review round, each article will receive one of the three following decisions: Accept with minor changes, Revise and Resubmit, or Reject. A R&R decision gives authors the chance to revise their submission and go through a second review round. We intend to improve the quality of CHI publication by giving authors an explicit revision cycle within the review process. R&R is also based on informed decision making and a balance of responsibilities for a fairer review process.

Before detailing the rationale for such a change, here is a short reminder of what the review process has been for previous years. Authors submit a paper. The relevant subcommittee chairs (SC) assign this paper to two associate chairs (AC), who become the primary AC (1AC) and the secondary AC (2AC). The 1AC assigns two external reviewers. These external reviewers, as well as the 2AC, write a review anonymously. The 1AC summarizes these assessments in their meta-review. Authors then write a rebuttal, answering reviewers’ concerns and making proposals for possible changes in a revision. At the PC meeting, both the 1AC and the 2AC know the authors’ identities when they decide: accept, reject, or accept with shepherding if the committee asks for more substantial changes. All authors of accepted papers revise their final submission. Shepherded papers have an early deadline for shepherds to check on substantial changes.

As detailed below, moving to a Revise and Resubmit process aims at further establishing CHI as a premier, high-quality venue and at guaranteeing more equity for authors in the reviewing process.

High-Quality Content

CHI articles are considered journal-level publications. An R&R process better matches journals’ reviewing practices. Authors benefit from feedback at the end of the first review round and get a real chance to propose a revised and better presentation of their work. We expect the process will lead to high-quality and impactful contributions.

Informed Decision Making

Possibilities for authors to address concerns raised in their submission used to rely on rebuttals and, sometimes, shepherding.

Rebuttals were promises made to the program committee in a non-anonymous way (the committee knows authors’ identities when they decide at the PC meeting). The committee had to estimate and speculate on how requested changes would be implemented in papers. Here, implicit and explicit biases (e.g., regarding authors submitting for the first time) can quickly shift a conversation. Seeing the changes already implemented by the authors provides the committee with a better ground for making informed decisions; the changes can be judged for what they are.

Shepherding relied on individual ACs. Shepherding a paper means taking on significant responsibility and committing to spend time with the authors on their revision. Some ACs might be more inclined/available to do so than others, meaning that authors’ chances partly rely on ACs’ personalities and own availability at the appropriate time. With an R&R process, evaluation of requested changes is a collaborative effort between ACs and external reviewers.

Balanced Workload and Responsibilities

Evaluation of post-rebuttal changes relied exclusively on the program committee (most often on the 1AC). With R&R, we expect the responsibility and workload of ACs to be better balanced, as the decision and evaluation of the revision are more evenly distributed across committee members and external reviewers. In particular, the second reviewing cycle includes external reviewers back in the loop. Those external reviewers are often the most experts to judge the revision in light of their initial concerns. That way, we can broaden our recruitment strategies for prestigious roles on the committees and enable more people to participate in these roles.

Some Questions

  • What kind of revisions are good candidates for a second round? As opposed to a journal reviewing process, R&R for CHI is implemented on a limited time frame. In particular, this means that reviewers are not expected to provide a second full review but should quickly evaluate minor changes. This includes: improving presentation, rewriting a discussion, moderate reframing, providing more details, expanding a bit on existing analyses. However, changes that could significantly change the contribution, such as running an additional study, are considered too significant.
  • Will ACs’ load be higher? We expect not, as the reviewing load is distributed among all reviewers, including external ones. We estimate the load to be equivalent to reading rebuttals and engaging discussions about what the final paper might look like based on the rebuttal.
  • Will there be papers rejected at the second round? Yes, there will. The R&R process aims at increasing the quality of submissions and fairness to authors, not at augmenting the volume of CHI papers (which is already very difficult to accommodate in a conference program). One might argue that a revision is more effort than a rebuttal and that rejection might be more challenging to handle for authors. This might be the case indeed. However, it is probably preferable to the frustration of not having the chance to demonstrate potential changes. Furthermore, our shared goal as a community is to build high-quality literature. A revised paper will make a better contribution for any venue, including CHI, next CHI, or another HCI venue.
  • Will there still be a rebuttal somewhere in the process? No, there is no rebuttal anymore. As mentioned above, no more promises, actual changes.
  • Will there still be a PC meeting? Yes, there will be a short PC meeting at the end of each of the two rounds. At the end of the first round, the committee will make one of the three decisions: Accept with Minor Revisions, Revise and Resubmit, Reject. At the end of the second round, the committee will discuss revised submissions and make one of the two decisions: Accept with Minor Revisions, Reject.
  • Can people get a visa to attend the conference with this timeline? For the round one accepted papers (a.k.a. fast-tracked), authors will have an extra month from the previous years’ timeline since their accepts will arrive early December. For the R&R papers, there will be more of a rush. To help offset that time crunch, we will be working on some new methods to expedite visa letters to help everyone speed up their paperwork process. We will rely on virtual and hybrid presentations and interactions for all authors who have visa difficulties.
  • How were the timeline and deadlines chosen? The ACM publication process takes an incompressible time, which determines the latest deadline in the process. Starting from there, we considered each phase in the process and set its duration by taking into account the workload of authors, reviewers, and chairs. Each individual deadline was chosen with the Equity and Justice chairs to avoid conflicts with major holidays around the world.
Categories
Uncategorized

Welcome to CHI 2022

Welcome to CHI 2022

Published: May 13, 2021 at 21:26 UTC

On behalf of our organizing committee, we’d like to welcome you to CHI2022.

CHI2022 will be held in New Orleans, Louisiana – and barring major changes in the direction of the pandemic we anticipate meeting in person. More accurately, we will be having a hybrid conference, where some elements will be online, and some will be in-person. It’s too early to clearly say what the hybrid experience will look like, but we are committed to making sure all elements are engaging and exciting for participants. Our phrase has been “no second-class experiences” no matter the mode you attend CHI. We understand that while many are anxious to get back to in-person meetings, many need online experiences in order to equitably experience the conference. Between traditional cost barriers, concerns about sustainability, and uneven pandemic recovery, we know that we need to be able to deliver a first-class online CHI experience.

Our slogan for this year is “Cultivating Communities”. One of the many lessons we’ve learned over the pandemic is how difficult it is to maintain and build our communities in a mediated context. We’ve also seen in the past year how communities can be unjust for some members. We approach “cultivating communities” with the idea that we have to be purposeful in building just communities that reflect the goals and needs of everyone in the HCI space. We know that CHI is a network of communities as well, and that building connections between them and the people in them requires communication work. We hope you do that work with us.

We’re excited by many of the initiatives we’re implementing this year. One is a focus on equity, justice and access. This year, Katta Spiel and Christina Harrington have been leading our efforts as part of the executive team. What that means is that they’ve been part of every meeting and every decision since we started organizing the conference – helping to ensure that our principles in these areas are reflected in all of our activities.

This year too, the papers program will offer an opportunity for Revise and Resubmit. There will be another blog post about the details of that, but in short we are excited about this new approach and see it as an equity and quality advance.

If you can make it to New Orleans, I hope you take the time to enjoy the city. New Orleans is a beautiful city, steeped in art, food, music, and history. Given its tumultuous history, it displays aspects of Indigenous, African, Spanish, French and American cultures, constantly remixing them. Enjoying a beignet at Café du Monde, or listening to jazz in the French Quarter, or touring the many art galleries – New Orleans has something for everyone.

We hope to see you there, as well as online for CHI2022. Let’s build and improve this community of global HCI together.

Cliff Lampe
Simone Barbosa
General Chairs

Caroline Appert
David Ayman Shamma
Technical Program Chairs

Katta Spiel
Christina Harrington
Equity, Justice and Access Chairs