AWS Case Study: Autodesk Uses AWS to Develop User Communities and Increase Community Participation
Software provider Autodesk has hosted a community forum called Autodesk Forums for its customers since 2000, but while many customers were using it as a resource, fewer were taking the next step in engaging with the community. In fact, the forum’s most engaged users were those best versed in Autodesk’s offerings. The company wanted to expand the forum’s reach by empowering customers to not only absorb community expertise but also offer their own.
Autodesk chose Amazon Web Services (AWS) solutions to build and rapidly deploy a machine learning model for a new iteration of a forum called Community Match. The model would match the expertise of forum members with questions posed on the forum — the idea being to encourage community members who are experts on a particular subject to share their insider knowledge on Autodesk solutions. And whereas previously customers sought Autodesk support only to troubleshoot problems, now they take advantage of the knowledge within the shared community to use Autodesk software more effectively.
Building a Forum to Encourage Community and Empower Customers
Autodesk makes software that enables people to “make anything.” Its solutions, featuring emerging technologies such as 3D printing, artificial intelligence, generative design, and robotics, are designed for builders in architecture, engineering, construction, media and entertainment, and manufacturing industries. The company has a strategy to principally use AWS for its entire development practice: in 2017, it moved its data science and machine learning development practice from on-premises machines to AWS, and recently it created a machine learning skills model to better route customers to support agents. “Similar to how we think about skills-based routing, we can start to serve our customers as part of the digital experience and in an intelligent way long before a person has to get involved,” says James Bradley, director of data science for Autodesk. “We can push our support capabilities closer to the customers and engage with them automatically much earlier.”
Autodesk originally intended Community Match to be a space where customers could connect with internal Autodesk experts. Later, in a shift to focus on nurturing shared expertise and community between its customers, Autodesk sought to increase customer engagement to help provide answers more quickly and an optimized user experience to more customers. “There was an opportunity to target members of the community who have specific expertise and pair them with members of the community who have questions that need to be answered,” says Bradley. To increase participation, the initial strategy was to use email notifications. However, that seemed to only engage customers who were already very active on the platform — people Autodesk called “Expert Elites,” whose product expertise makes them de facto Autodesk ambassadors. Autodesk instead wanted to use the notifications to draw in a new set of customers who had a demonstrated history of participation and interest in the forum but didn’t regularly participate.
To gain insight into how customers would react to email notifications to drive forum engagement, Autodesk went straight to the source, interviewing customers at an Autodesk conference with over 10,000 attendees. A major finding was that some customers’ staff members actually scour the forums or ask questions to curate content to share internally with their teams but don’t always share expertise themselves. “Many of them feel that they don’t know as much as an Autodesk employee or an Expert Elite, so they wait for someone else to respond to an incoming question — even though they may be subject matter experts in that field,” says Yizel Vizcarra, data scientist on the digital help and experience team. “We wanted to encourage a sense of community in the forums and encourage people to reply when their expertise can help another individual.”
Developing Creative AWS-Backed Solutions to Serve Customers
Autodesk used an AWS serverless architecture to create a prototype of the knowledge model in just 1 week. “It was fun to create something from just an idea in such a short time by plugging together the different serverless capabilities that exist on AWS,” says Bradley. A webhook delivers real-time data to AWS from the third-party vendor where Community Match is hosted. “It provides a flexible architecture for us because we can carry that content in many different ways, not only keeping employees in the loop by sending things to Slack but also catering to our different user groups,” says Vizcarra. Autodesk split the forum users into groups: highly engaged, semi-engaged, and observers.
Autodesk then built and trained a knowledge model, hosted on Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service, using a transfer learning technique to create embeddings of customer questions from the forums. Then it built an approximate–nearest neighbors model — a common classification model based on the assumption that items close together in a dataset are typically similar — using Amazon SageMaker. The knowledge model and approximate-nearest neighbors model help analyze incoming user questions and pinpoint 10 other users who have the expertise or have answered a question on a similar topic in the past. The models and a series of business rules are packaged using AWS Lambda, which enables Autodesk to run code without provisioning or managing servers, and sequenced by AWS Step Functions. These notifications, known as recommendations, are sent using Amazon Simple Email Service (Amazon SES) to up to 10 users per incoming question with a link to participate. Autodesk built the entire solution without changing its existing software infrastructure. “We were able to introduce machine learning for real-time monitoring without having to do an extensive integration with the forum software,” says Alex O’Connor, lead data scientist for the digital help data science team. “In fact, we relieved pressure on our servers by doing this.” Without streaming, it would have been necessary to execute very large bulk export queries against the forums, which have enormous resource costs, potentially affecting the forum platform experience for users. With the real-time webhook approach, the resource scaling is easily controlled.
A new Community Match iteration launched in July 2020, targeting customers who do not regularly participate. In the first 6 weeks of the forum’s launch, Autodesk sent 8,473 recommendations using Amazon SES and Amazon Simple Notification Service (Amazon SNS), a publish-subscribe (pub/sub) messaging service that is used to create topics or logical groups on different types of products and delivers update notifications. Autodesk found that the machine learning model makes a high-quality match 32 percent of the time, which Autodesk judged by the open rate, and the click-through rate of notifications was 31 percent. Compared to a 12 percent reply rate in the first iteration of Community Match, at least 16 percent of people who clicked on the notification in the new iteration — the majority of whom were previously not highly engaged users — then engaged in a reply. “Receiving a notification that emphasizes their expertise hopefully encourages them to fully engage in a conversation,” says Bradley. “And in fact, we’ve seen them posting a little bit more.” The forum is designed to empower customers — particularly those who refrained from participating — by showing them how sharing their expertise can benefit others. Already customers have offered off-book answers or workarounds different from but no less helpful than the expert advice that would have come from an Autodesk agent.
Autodesk can also use the forum to better engage with and help its customers. “We can react to customers behaving in real time and drive positive business impact by doing so,” says Bradley. “It opens up opportunities to think about our business goals: we could build on top of this real-time messaging framework to drive customer behavior that reduces their effort or increases business efficiency.” For example, the forum has given Autodesk the flexibility to rapidly respond to customers’ needs in a way that a campaign manager or email application could not. During the COVID-19 pandemic, for example, it repurposed pieces of the forum infrastructure to monitor mentions of the coronavirus to note the business adjustments that customers were making, and from that data it developed informed responses. “Having an ear to the ground is great to support responding in real time,” says O’Connor. “Especially early on, when the situation was developing so rapidly, it was important for us to understand whether we were reaching our customers’ needs at the same speed.”
Listening and Quickly Adapting to Customers’ Needs
Without making in-depth changes to its existing infrastructure, Autodesk used AWS services to reconstruct its Community Match forum to drive more customer engagement, empowering customers to share expertise that others could benefit from. The machine learning–driven forum not only enables Autodesk to creatively deliver answers to its customers but also provides the company with valuable customer insights and a flexible solution that can quickly adapt to customers’ needs. “We really do have the ability to customize it and learn from it very quickly,” says Bradley. “It’s about listening to how we can help and adapting our response to that.
