The ACM CHI Conference on Human Factors in Computing Systems is the premier international conference of Human-Computer Interaction (HCI). CHI – pronounced 'kai' – annually brings together researchers and practitioners from all over the world and from diverse cultures, backgrounds, and positionalities, who have as an overarching goal to make the world a better place with interactive digital technologies.

CHI 2022 is structured as a Hybrid-Onsite full conference from April 30–May 5 in New Orleans, LA.

Hybrid Conference In-person conference

Making the First Hybrid CHI in 2022

Making the First Hybrid CHI in 2022

In the months since CHI 2022, “Running a hybrid conference is essentially like running two conferences” was the echoing comment I’ve heard on hybrid conferences. The error here is if one designs two conferences, then they do not have one hybrid conference. For CHI 2022, our goal was to have one blended conference. As researchers in the field, it is easy to have the “what does hybrid mean” discussion and to retell accounts of that great hybrid conference you went to with less than 57 people. None of this matters at scale, assembling a handful of Zoom rooms and a Slack channel doesnt work when you have hundreds of authors and thousands of attendees. A few people have asked us how did we build a hybrid conference, what worked and failed, and should be done differently; this article aims at detailing all of that and does step a bit into the technical weeds.  

First off, we were thinking about boots on the ground when planning. Here we focused on a few core deliverables:

  1. An architecture that utilizes SIGCHI infrastructure and that is reusable.
  2. Video pixels need to be streamed out clean and clear.
  3. People should have one place to go to consume content (locally or onsite).
  4. Blended interactions are essential.

And while we want to do all we can, we must stick to delivering the minimal viable product (MVP) first when designing the experience. I’ll speak mostly to the design of these systems and leave out the gritty bits of the vendor discussions but will include some of that where needed.


Having spent some years as SIGCHI’s VP of Operations, I was familiar with many of the systems in place. Stitching them together was possible but our SIGCHI Programs App would just have outbound links to the video stream, the slido for Q&A, the discord room for chat, the paper in the DL, and wherever else needed. In hindsight, I wish we did this, but we wanted to minimize the application windows and logins (trying to only use our registration system) to the best of our ability. The Programs App was not able to host embeds yet, which would have solved the task at hand. As an alternative, we picked a vendor (HUBB) which could host the page with the embeds and promised to deliver a WCAG 2.1 AA accessible experience while doing so. Lastly, this all has to be linked with presenters (remote and local), AV staff, and attendees (remote and local). In the end, we had the following flow:

A workflow that goes from the left to right with local/remote presenters to a mixing board which forks to a in room projector and a YouTube stream embedded in a Hubb page. Viewer entry point is the PWA on the right of the diagram.
Figure 1: CHI 2022 Hybrid Architecture

Here we see the audience can enter the experience using the SIGCHI Programs App (PWA) and navigate to Discord (for chatting with local and remote people) or get to the Slido for questions or to the livestream if they aren’t in the room. People presenting are all funneled through Zoom (no more plug in your laptop) which is projected to the room and streamed out via RTSP. If someone couldn’t make it (maybe they got sick or had technical issues), we had the pre-recorded video presentation for backup. These pre-recorded talk videos are also persisted and stored with the program (and are often reshared and viewed after the conference). This architecture was optimized for streaming pixels, reducing platforms, and hosting blended interactions. While some of this went great, not all of it went to plan.

Streaming Pixels

In many ways, this is the obvious core to any hybrid event. If remote people can’t consume the content in real time, then it’s just an asynchronous viewing party. While not uncommon to the Internet, live streams in our use case require a fair bit of infrastructure. First, we need a high quality feed so the slides are viewable. Second, we need that feed with closed captions delivered. Third, if someone is remote, they should be able to ask a question into the room. Finally, it would be nice to see the speaker, the audience, and the room. With regards to the MVP, that last point can be omitted if it has to be.

As we learned the core of this falls into the hands of the onsite AV vendor. You know these people; they are the ones who tape down microphone cables and set up booms at conferences you probably attended before the pandemic. They also set up cameras and mixing boards and manage streams. The connection to the Internet (and bandwidth therein) usually is under the jurisdiction of the convention center. AV and Internet account for the bulk of the added cost for hybrid; given that the SIGCHI EC did not allow for an increase in registration for 2022 (and CHI registration has increased in total by 40 USD in the last decade with no increase in the past 6 years), it is arduous to cut costs and add hybrid experiences. To go hybrid adds hunderds of thousands of dollars for each needed piece: AV and Internet, the virtual platform, and added project managment costs. If your curious official numbers will be posted over on SIGCHI’s site when finalized; also historical budgets are there already.

Referring to our architecture in Figure 1 above, if we have 4–5 speakers in a session, with two remotes we can set up a very complicated mixing board to read the HDMI feed from the slides in high def and spot a camera on the speaker with one or two people working the mixing board and the camera. For remote presenters, they will come in via a video conference tool anyhow (we used Zoom). So to simplify the rig, we had all attendees (local or remote) present through Zoom and that direct line was on the mixing board and broadcast to the room. The shuffle to change HDMI/DVI/VGA drops is replaced with presenting in Zoom and despite noting the change in pre-conference correspondence, presenters gave significant push back asking why can’t we change it back to the old way of just using a cable. A wide view stage camera takes the place of a camera operator and can capture some of the room and speakers. What’s happening is we are simplifying the configuration and human power needed to run this event (and saving costs as to have a full crew on site is prohibitively expensive).

Unfortunately staffing issues during the pandemic made it difficult to find trained AV experts. Being sandwiched by New Orleans Jazz Fest also didn’t help as the experts available were working (presumably) higher paying gigs. Many of the issues we faced on day one happened right at that mixing board: double audio streams, switching inputs between pre-recored video and Zoom, etc. In one session, I walked over to the mixing board to drop an audio level. We can’t blame the operators here; it’s a byproduct of recovering from a pandemic; still it was an issue we faced.

One Location for Consumption

The last thing we wanted was to hand out 3–5 URLs for each session for people to use for watching, chatting, asking questions, seeing who’s speaking, finding the paper in the DL, supplemental video and whatever else. The SIGCHI Programs App does a good job here for detailing the content and hosts several links under each session or paper. We wanted just a single landing for all of these interactions embedded so people didn’t need to go between platforms. The SIGCHI Programs App didn’t have the ability to handle the embeds at this point but that is and should be the solution for all future SIGCHI events. I’m working with the SIGCHI VP of Operations to make this path happen.

Now there’s a whole industry for this where we see Hubb (which we used this year), Delegate Connect (from last year), Midspace (formerly Clowder used at other SIGCHI events). I’ll personally go on record saying these all don’t meet the standards we set for CHI and SIGCHI. First of all, we have a superior program app. Second, none of them (really none of them) adhere to our accessibility standards (despite that they go on contract saying they do comply). Third, there’s a fair bit of effort duplication to stand up these sites once we have the SIGCHI Programs App in place as the program has to be replicated elsewhere. It’s a mess.

Unfortunately, the path was turbulent from the moment the vendor contract was signed. Much work had to be duplicated by the volunteers (migrating from the Programs App to HUBB). HUBB makes more logins for people. HUBB’s chat (for Q&A) was not accessible to our standards nor patchable so we had to (last minute) use Slido. HUBB’s navigation was overall poor compared to the SIGCHI Programs App. The list goes on. The “let’s have one platform” idea did not work.

But something more was happening here beyond one vendor not meeting our quality specifications. I realized by watching attendees, we don’t live in a one platform world. Stop right now and count how many tabs are open in your browsers (yes plural). We are used to juggling platforms. How many times does someone check Instagram or Twitter on their laptop versus their phone? Some people prefer to watch a video on their iPad and type questions into slido on their phone. SIGCHI’s IMX conference grew out of research on second screen experiences. The community dissected the Hubb platform and started sharing slido room numbers and the direct YouTube live stream links. Many of our attendees were sharing the Slido rooms with others on Discord (so much that we wrote a script to broadcast the Slido room numbers there). Others were sharing the direct Youtube stream links openly as the HUBB embed was prone to failure. This was great as we made the decision to just open up all the live streams to everyone (registered or not) so they were easily viewed. I know I was asking slido questions from the slido app on my phone with discord up on my iPad. It’s the Zero, One, Infinity rule in effect. One platform won’t work and people will want to consume content in native environments or just pick the one piece they want. Embrace the platforms and provide access to the atomic pieces.

Blended Interactions Are Essential

Being hybrid is not about segregating the online crowd into their own channel to discuss amongst themselves. Hybrid is about blending interactions. If you take live questions from the audience and have a volunteer in the room scanning an online chat room to find and surface remote questions, the remote attendees will basically be ignored. Here running all questions through Slido allowed everyone to put in a question and allowed the session chair to have a single feed to scan for questions to relay to the speaker. Now we will miss the occasional person who would take the mic and launch into a soliloquy on “why you didn’t cite my past work?” but personally I’m ok with that. What does get lost is people finding out who other people are. We could start a practice of people listing who they are in the question: “Why didn’t you cite my past work? (ayman • CHI22 TPC)” to help. You could probably try to enforce a one live question one remote question protocol but as any CHI organizer will tell you “enforcing anything at CHI is next to impossible”. Also there’s no promise that the presenter will be in the room anyhow since we now allow remote presenters.

Remote session chairs just didn’t work. Initially we didn’t want this feature but an equity argument was made and we didn’t have enough session chair volunteers. Having remote session chairs didn’t work because the first job of the chair is not to meet speakers and ask questions if there’s dead air after a talk. The session chair is to control the room (take the conn nautically speaking). This is impossible to do when remote. After day one of CHI we worked closely with our student volunteers and retrained the AV staff so things went smoother but in effect wasn’t perfect. I think remote chairs could work if we brief the SVs for those sessions in advance advising them how to take the conn. There is the cost of the added burden on the SVs to take on this role, but in situ at CHI 2022, many SVs did so to save the day. Thanks SVs you all are amazing!

Some venues just will need bifricating: most notably Interactivity. Watching a livestream of other people using a demo is less compelling than just watching a polished video of the demo and chatting with the authors and builders. Once upon a time there was a CHI video theather where we’d have popcorn and watch video demos. Perhaps bringing that back in some community chat space for live discussion might be the solution. Which brings me to perhaps one of my favorite things about CHI 2022: Discord.

Initially I’ll admit I didn’t want a Discord server (as I was focused on fighting platform creep). When weeks before launch it turned out the chat rooms in Hubb were not WCAG-2.1 AA compliant we added Discord (lead by Monica Pereira, Danilo Gasques, and Kashyap Todi who did amazing work setting it up, vetting attendees, and writing bots). Discord became the mechanism where people on site and people who were remote connected, chatted, complained, praised, and troubleshooted. It was kinda cool to see how quickly the community bloomed there. Pretty much the few voices I heard complain about Discord onsite just rejected using it or logging in.

Now at this point, you might say ‘Why not just let anyone ask a question and put all the remote attendees in a livestream room and nothing more?’ This seems simple but ultimately won’t work. First, to interact with the sessions (even if just asking a question) one would have to be registered for the conference (even if cheap or free) as that puts the person under our code of conduct. Second, the livestream platform would have to only allow those registered people in which would create another point of login or new logins to create for people (sound familiar?). Lastly and most importantly, this effectively walls off the remote viewers away from the local in-person attendees. The goal should be to blend spaces and people, not segregate them.There is the issue of the attendee who is onsite and “can’t be bothered with Discord” to which I’d just say the world has changed and it’s their loss; they get the lesser experience by not joining the discussion.

What About Future CHIs?

The good news here is the SIGCHI Programs App is now equipped to handle embeds too, so future conferences can offer a mostly “one login” experience as well as provide the outbound links to the individual platform tools. Many of the pipelines in our architecture offer reusable and easily scriptable datastreams. I’m working with the current SIGCHI VP of Operations (Hi Kash!) to build out a flexible path here. Not pictured in the architecture above is the SIGCHI QOALA app which links to PCS (our paper review system) allows conference chairs to design the sessions and layout the conference. If QOALA creates the embeds, live streams, and q&a rooms programmatically, then hundreds of hours of volunteer time can be saved. For example, imagine a talk is set for Monday morning but during planning that talk is moved to Wednesday afternoon. All the links and rooms and embeds around that talk have to be updated (and yes we did this manually in 2022). If QOALA can stand up the embeds, it can keep track of schedule changes and update things automatically. This architecture would take a new simplified form as seen in Figure 2. This would require only two logins: an ACM login and a Discord login. Both we can ensure will adhere to our privacy standards and the Discord can be shut down after the conference.

A workflow that goes from the left to right with local/remote presenters to a mixing board which forks to a in room projector and a YouTube stream embedded in the PWA. Viewer entry point is the PWA on the right of the diagram.
Figure 2: An ideal architecture, doable for future SIGCHI events, where we use our own web platform. Dashed line denotes possible embeds in the PWA to YouTube Stream and Slido.

In the early days of the pandemic, as SIGCHI VP of Operations, we worked hard to stand up Asynchronous Hybrid (with RecSys being the first on deck). Two years ago when we started planning, myself and Caroline Appert (the TPCs), and Simone Barbosa and Cliff Lampe (the GCs), and the rest of the CHI 2022 committee believed the path forward for hybrid is synchronous. With the successes and obstacles we faced in 2022, I still believe synchronous hybrid is the future. Moreso, by working with the EC on this solution, it allows all 24 SIGCHI conferences to follow the same mechanism and lets SIGCHI become a leading example for hybrid conferences with blended remote & local interactions.





General Program Hybrid Conference In-person conference

CHI2022 WrapUp

After a few weeks, we take the opportunity to look back and reflect on the CHI experience. 


More people reported cases of COVID infections than we had hoped. We always knew COVID infection was a risk at the conference, and took the steps we could – including having a strong online option for the conference.

As of writing, 127 people reported positive cases to the ACM. Of course, that number is likely higher – given that people may not have reported. Without asking people to report negative tests as well, we can’t really even generate a response rate. At the end of the day, it doesn’t matter if we’re talking about a 5% or 10% infection rate. Anyone getting sick is bad, and some realized their worst fears about in-person conferences. For everyone, we’ve reflected deeply on every cough, every moment of anxiety, and every hour stolen from already absent loved ones.

We could go back and forth about the steps we took to prevent COVID at the conference, or where the infections stemmed from. We’re not epidemiologists, so we’d be bad at that. There are really only two questions that matter: Should we have had an in-person conference? What should the next in-person conference do?

In terms of whether we should have had an in-person conference, we stand by our decision to have an in-person version of the conference. Everyone should be able to make the choices to protect their health without sacrificing their careers, so we need to put more thought into meaningful online experiences.  For advice for future conferences, it’s important to think about that hybrid experience and how the online experience creates value for people. In terms of future conferences meeting in person, we hope they have less infection than we saw. However, this may be the world we live in for a while – where we have to do our best to protect the safety of people in person, while realizing we have limited abilities to control outcomes.

How Hybrid Went

Our goal this year was to privilege a form of hybrid that emphasized synchronous presentation and interaction. We also wanted to allow both presenters and audience maximum flexibility, as we realized people would be deciding often at the last minute whether they would be in person or not. As with all decisions, we also prioritized accessibility as one of the major criteria that our system should have.

Frankly speaking, nothing worked as well as we hoped. The system was too clunky, and the flexibility we hoped for from AV turned mostly into points of potential failure. What worked and what didn’t? 

  • WORKED: The SIGCHI Progressive Web App. This app worked consistently and well. With some minor additions, it really could be a center for a conference’s hybrid strategy.
  • DID NOT WORK: Online platforms. People often got confused if they were in Hubb, or Zoom, or what. We had hoped we had found one vendor that would create a single online experience, but when that didn’t work, we had to pivot to other solutions in the moment to make our interaction goals work.
  • MIXED BAG: The AV set up. The AV industry has been hit hard by the pandemic. In addition, to save money we had hoped to use less staff and more volunteers. Overall, we spent 3x more on AV than any previous CHI – which is necessary because of the synchronous hybrid goal. Things got better quickly, but it was a rocky start.
  • WORKED: Discord. While it was “yet another platform”, it was free and worked consistently well.

There are teams at both the CHI Steering Committee and the SIGCHI Executive Committee post-gaming hybrid experiences and trying to make recommendations for the future. The scale of CHI is a challenge on many levels, but there are some consistent issues all conferences are facing.


This is an issue that surprised us – breakdowns in how we treated each other.

It’s interesting to think why this is. To some extent, norms have changed over the pandemic, often for the better. For example, people are more willing to say something if they feel uncomfortable. People have a better sense of what harassment is and feel more like they can report when conversations cross a line. That’s all good.

Another effect might be the general anxiety we feel from the ongoing pandemic expressing itself on underserving targets. The student volunteers were often the targets of bad behavior in the rooms they were helping to support. That’s never acceptable. Our hope is that this was unusual behavior shaped by stress, because we will not ever condone bad treatment of our SVs.

While some norms have seen positive change over time, some have changed for the worse. One of those we saw was the tendency to state things in sensationalist or absolutist terms in order to receive approbation. We see this all the time on social media, where signals of attention are poor so some people post the most extreme version of their message in order to get a reaction. 

Everyone deserves to feel safe at the CHI conference. No one should feel demeaned because of their country of origin, their research area, or their institutional affiliation among other things. While it’s a good thing that we raise critical points about issues in the tech industry, it’s possible to do so without demonizing or trying to score points. Whatever issues one has with any institution, the people at our conference who are part of that institution still deserve to feel safe and not demeaned. We will work better in the future to help our keynotes understand these values.

We continue to grow and learn to act better. This year tested and executed on our anti-harassment policies, but in that test we saw positive outcomes. Justice and safety for our members is a process, but we are encouraged by what parts of that process we saw.

Survey Results

A total of 749 people responded to the request to fill out the post-conference survey. 582 reported attending in person, 148 virtual, and 19 didn’t report their mode of attendance. This means that response rates for the virtual mode of the conference (~8%) are quite a bit worse than for those who attended the in-person mode (~29%). We don’t know why the response rates are so different.

A bar chart showing the frequency of Likert category responses to the question “How valuable was your CHI experience?” The pattern is that most people reported a high value in the conference.

One of the most essential questions we ask, and really the essential dependent variable we ask year-to-year is “How valuable was your CHI experience?” This is measured on a 5 point Likert scale where 1 is “Not at all valuable” and 5 is “Very Valuable”. When we collapse the modes of the conference, overall the ratings were fairly high. The median score is 4, and the mean is 4.27. The bar chart below shows the overall pattern.

A bar chart showing the frequency of Likert category responses to the question “How valuable was your CHI experience?” The pattern is that most people reported a high value in the conference.

The numbers look different when we compare the virtual and in-person modes. 


A bar chart showing the frequency of Likert category responses to the question “How valuable was your CHI experience?” The pattern is that most people reported a high value in the conference.


Alt text: a bar chart showing the frequency of Likert responses for in-person participants.

Descriptive StatsIn-PersonVirtual
N (Valid)576148
N (Missing)60
Mode 5.004.00

As usual with CHI, people value different parts of the conference. The chart below shows different ratings by different parts of CHI. It’s impossible to experience ALL of CHi, and in general people seemed pretty happy with the parts in which they participated.

 A series of stacked bar charts showing likert scale ratings for each technical program at CHI2022.

We asked people about how CHI2022 dealt with COVID-19. In general, people felt like the organizers did a decent job with the measures in place.

Q9. Do you agree or disagree that organizers adequately considered COVID-19 protocols at the convention center?

A horizontal bar chart showing likert responses to the question “Organizers adequately considered COVID-19 protocols.” The overall pattern is either agree or strongly agree to that statement.

In the open-ended responses, two possible different measures were mentioned. One was to encourage daily testing by participants. The second was to not offer food at breaks and receptions, given that sharing food obviated to some extent the benefits of masking. Participants also felt like they weren’t as careful as they could have been.

Q11. How informed did you feel regarding the conference’s COVID-19 protocols before attending the conference?

A horizontal bar chart showing the response frequencies to the question “I was informed regarding the conference’s COVID-19 protocols.”

There’s a lot more to the survey, all of which is being considered by the CHI Steering Committee and SIGCHI Executive Committee. For instance, having a cash bar was controversial. Some people loved it, some people felt like it didn’t go far enough in removing alcohol from the conference, and some people hated the move as punishing all attendees while not really preventing harassment. 


In summary, there are some things we’d like to change about how CHI2022 happened, given the clarity of being after the fact. However, we are also proud of the event that happened given the uncertainty, complexity and sheer difficulty of putting the event together during the pandemic. We want to end by appreciating the incredibly hard work of all of the volunteers who put time into the conference. This was very much like organizing 2.5 conferences, and it could not have happened without extraordinary effort by our organizing team.

Technical Program

Moving to a Revise and Resubmit Review Process for CHI 2022

Moving to a Revise and Resubmit Review Process for CHI 2022

As an effort to improve paper quality, increase equity, and reflow the reviewing process, CHI 2022 Papers Chairs, Technical Program Chairs, and General Chairs have replaced the paper rebuttal mechanism with a Revise and Resubmit (R&R) reviewing process. R&R is a two-round reviewing process. At the end of a first review round, each article will receive one of the three following decisions: Accept with minor changes, Revise and Resubmit, or Reject. A R&R decision gives authors the chance to revise their submission and go through a second review round. We intend to improve the quality of CHI publication by giving authors an explicit revision cycle within the review process. R&R is also based on informed decision making and a balance of responsibilities for a fairer review process.