Case study: User research on one of the most active & popular CBCs in India
This case study looks into the research process followed to understand & improve an online community experience. A 10k designers production!
The Mission-
Should you choose to accept…
- To understand how online communities function and interact on a daily basis and notice behavioural and usage patterns.
- To understand their purpose and what they provide to their members.
- To understand the process of user research, the various methods available, how to approach them and collect useful insights.
- To identify pain points of the community and suggest design solutions, that, if implemented, will improve their overall experience.
The team-
Grateful to be a member of an awesome team, comprising Sivaprakash, Prerita Yadav, Arnav Sharma and Nisheta Gupta.
Despite having classes, exams and full-time jobs, we stayed up late, conducted interviews and made sure we were motivated and filled in for each other, thus, keeping us going. So Kudos team!🎉
Let's begin our journey into the world of Ed-Tech and community learning shall we?! *cue theme music*
An open-ended brief
Unlike regular briefs with constraints, we were given the freedom to choose a niche, and community to research. It just had to be an online community.
After some dedicated brainstorming sessions over what possible niches and communities we could go for, we decided on going for Cohort Based courses (CBCs).
We also looked into many existing case studies, various user research and design thinking methods before finalising our approach for this research.
Before you ask, as to what made us zero down on CBCs, let us jump into a flashback for some context…
The flashback- What and Why CBCs?
To understand CBCs, we need to understand the Ed-Tech foray and how education on online platforms has grown over the past years. For a better insight into how online education has evolved manifolds, the diagram given below is indicative; the most prevalent structures are the MOOCs and CBCs.
MOOCs are online courses that a student accesses through the internet. Typically, these courses consist of traditional class materials, like, recorded lectures and reading material, made accessible online.
CBCs are collaborative learning styles in which a group of individuals advance through an educational program together. In this setting, you complete an educational program with your peers in the same way, but watch lectures and complete assignments online, instead of in person.
Here is a comparative study to provide a better understanding.
So coming back to our first question as to why we zeroed down on CBCs:
- Relatability factor associated with it, as we ourselves are a part of a CBC structure.
- Given their collaborative and community-based approach, we also felt researching CBCs would be extremely beneficial as in our view they may emerge as one of the leading Ed-Tech in India.
- CBCs have an accountability factor associated with them as you are made solely responsible for your progress and success.
- CBCs are a great place to network, reach out to potential hiring personnel, make friends and receive constant mentorship and feedback. Thus it adds value to one’s circle and improves one’s reach and visibility in the field.
Our ultimate endeavour -
- To understand how CBCs function as compared to the usual self-paced courses and traditional educational systems.
- To study the community aspect of CBCs in-depth, to bring out how it contributes to the learning process.
- Do some value addition for improving the process of online community learning and upskilling.
The target- Identifying the community
Finalising which CBC to research was our next milestone. We looked into a variety of CBCs and did a quick surface-level comparative analysis to make our choice.
A few pointers that were considered-
- How many active members does the community have?
- Ease of reaching out to members? Is it an open group?
- Social media presence (For collecting secondary information)
- The value they add to the investment done by members.
After a lot of contemplation, we finalised NeoG.Camp as our target community. Stay tuned for more on why they were chosen and how we went ahead with our research…
Extracting data- Secondary research
To start off, we looked into NeoG.Camp and the kind of experience and learning they provide.
Founded by Tanay Pratap and Tanvi Priya, this community set sail in late 2020 and has been growing rapidly ever since. They provide a platform for people from all walks of life, to program and be industry ready in a short span of 6 months; it's just a matter of interest and putting in the effort.
The beginning :
The #TeamTanayJobChallenge started in March 2020, where one had to finish a minimum of 5 projects, 3 blogs, and 2 reviews on their projects, before June 2020. The qualifying people would then be provided with an interview opportunity through jobchallenge.dev (Changed to Elevate Labs, now Roc8 careers). A good number of people completed the challenge and it got 15+ people placed in various companies, even during a pandemic!
This process, though brought to light, two major issues- The lack of skillset to complete the given challenges effectively and the readiness to face a job interview. Thus Neog.camp was born!
Creation over Certification. Craftsmanship over Credentials.
- Neog.camp
Social Media presence:
By having an extensive presence on platforms like LinkedIn, Instagram, Discord, Twitter, Telegram and Youtube, NeoG.Camp boasts of having the biggest active community for programmers in India. The founder, Tanay, himself has an extensive presence, making him one of the people many aspiring programmers and developers look up to, thus providing them with a reason to join the community.
Being the mentor I never had!
-Tanay Pratap, Founder, Neog.camp
How it is going so far-
The camp is designed in 3 stages-
Level zero- Free and open for all with no registrations. It is beginner-friendly and requires no preparation
Level one- A structured six-month BootCamp for the selected programmers. Entry is open only for people completing the Level zero submissions and qualifying the interview round.
Placements- After finishing level one, the selected candidates are given an opportunity to attend mock interview calls and get equipped and prepared before sitting down for the actual hiring process.
Get on the field- User interviews
Before starting off with interviewing and understanding the users first hand, we had to come up with our “script”- The questions.
We started on a general note through Instagram surveys and Google forms (Shared publicly and on our CBC community discord server), asking the public about CBCs and their knowledge of it. With 100 responses in total, it definitely gave clarity about the fact that CBCs are still a very new concept in India, but people are willing to adopt them.
The generic survey was mainly done to get an overall understanding of CBCs and the public’s opinion of it.
From here, we moved on to try and understand the NeoG community, how people came across this, what were the driving forces for them to join this community and much more.
To make sure we covered as many aspects as possible, we undertook the method of Brainstorm-Accumulate-Eliminate
After these processes, we came up with a set of questions, that acted as guides for us to ask and improvise as and when required. We reached out to potential interviewees through cold messaging on various platforms like LinkedIn, Twitter and Discord. Our main aim was to make the interviewee comfortable, keep the conversation flowing organically and be good listeners.
We also came up with a set of hypotheses based on all the research done so far, which we would validate post the user interviews and primary research. More on this in the sections to come. To be continued…
Recognise key members- User Personas & Journey maps
We came across 4 major recurring personas post our primary and secondary research.
User persona 1: Looking to upskill and land a better job
Current Status: Finished Level 0 submissions and awaiting a callback post-interview
User persona 2: 1+ year in the tech domain, directly interviewed through intermediate route for level one.
Current Status: Got selected for level one. Cohort starting from Jan ’22
User persona 3: 2+ years in the tech domain
Current Status: Neograds (alumna) from the first cohort.
User persona 4: College graduate, Fresher.
Current Status: Neograds from the first cohort, got placed at Unicorn startup.
Identify problems- Pain points
There are a few pain points that many of our interviewees mentioned, which are as follows-
- Given the sheer volume of the community, 1:1 discussions with mentors and team captains[*] are very minimal.
- Queries tend to get lost in the wave of messages and updates on Discord. The feedback loop is very disorganised due to the volume of the community for level 0.
- It becomes difficult to manage the cohort and its projects, along with a full-time job/ studies or when you don't have the background knowledge and are shifting fields.
- There is no organised space for study material, for us to go back and refer to, apart from an open-source like YouTube.
- There is a lack of a fixed learning schedule for level 0 since its self paced.
[*] Team-captains are assigned with a group of 18 members in Level one. A team captain is a go-to person in case of a query or a doubt. They are selected on the basis of their skill set. They hold the responsibility of the group and motivate members to finish assignments on time.
Eliminate effectively- Possible solutions
Following are a few solutions that we brainstormed as a team and came up with. Since the main platform that's used by NeoG.Camp is Discord, we have come up with solutions that cater to that platform’s specifications and structure-
1. No spam during Famjam
Since entry is open for everyone, there is always a possibility of spam and unnecessary messages, that take away the importance of queries posted. This problem persists, even when there is a separate server allocated for queries. Thus, there can be authentication provided for writing queries on these servers, and access provided only to these chosen members to post clarifications.
2. Questions? We “bot” you!
Piggybacking on the previous problem, A bot can be introduced to repost unanswered queries for a set time frame of 24–48 hours. Once the queries are answered, the person who asked them can close them automatically.
The bot could also help one raise a request to the person who has clarified the query, to get on a 1:1 feedback call, to which the person will have the freedom to schedule based on availability or reject with reasoning.
3. Karma isn't a b**** after all
In the existing system of discord servers, there is also a gratitude server channel, which posts a “thank you” message, for each query answered by peers.
Taking inspiration from Coursera and Interaction Design Foundation, we felt this system can also be gamified, by providing incentives, in the form of Karma points or badges, which would also promote peer learning and improve doubt clarification.
4. An organised common platform
[Update: A common platform is in the pipeline, which is being designed by the Neograds (members of the cohort) themselves]
At 10k designers, we use a common platform called Noggin, to access course material, upload assignments and reach out to mentors and alumni members on various platforms to network.
Currently, at neoG, the recordings of the live classes are shared with team captains in Level one, which in turn is shared with other members of the group.
We felt a common platform, similar to Noggin, can be designed, in order to organise all materials and resources provided, and also streamline the submission process, given it's a very important task for a member to qualify for the next level.
One could also link details like social profiles and portfolio websites of past cohort members, for ease of networking and reaching out for mentorship.
Currently the brand statement of NeoG.Camp is synonymous with its founder, Tanay Pratap. Crediting mentors and members, will not just provide good branding and “proof of success” to the cohort, but will also provide exposure to members along with more networking opportunities. This in turn will improve enrollment numbers and increase NeoG.Camp’s reach.
5. Level up, but realistically.
Currently, the members at level zero follow the same route, comprising of the same 15 assignments, even though the skillset may be very different for different people. As clarified earlier, level zero is open for all; college students, recent graduates, working professionals, people with prior work experience and looking to upskill or people who are planning to switch fields.
Thus, a skill set test can be done to gauge a person’s ability and knowledge in the field. Further, people with similar skill sets could be grouped together and assignments could be customised accordingly for each group.
This would ease management of time, along with a job or college work, and improve completion rates and quality of the projects and the overall cohort as well.
Confirm and launch- hypotheses validation
Following are a few hypotheses, that we had set forth throughout our research, which we validated at the end of the entire process.
That's a wrap folks: Conclusion
As always, mistakes are made, lessons are learnt. My journey along with my team members, during this assignment, was one filled with a lot of fun, a lot of exploration and a lot of learning.
Active listening.
Listening is a highly underrated skill set. The user interviews dint just help me become a better listener, but also helped me value silences and pauses. There were a lot of insights that I could derive out of the users, in between these pauses.
Working as a team.
As 5 different individuals, all of us had varied opinions and thought processes. But I learnt to accept and ruminate on the various opinions put forth. I started valuing the variety of ideas that can be brought to the table by a team, and the number of angles by which a problem can be approached.
Document. Everything.
Documentation is a very valuable skill that I learnt to inculcate for all my projects. Using Notion or just a simple notebook to write down all our findings helped us with keeping tabs on our progress and avoiding missing out on updates and ideas. It also helped us plan out our group calls and discussions based on the level of progress.
Post credits:
If you made it here, and are planning to stay, you are a true fan. I appreciate and respect your will to actually go through my entire study. Means a lot!
Firstly, Thank you very much 10kdesigners, for handing us this brief and providing us with great mentors like Vaishnav Ramayan and the material to start off with our research study.
Thank you Jash Valiya and Agrata Patel, for providing us with constructive feedback, even at 1 am and helping us streamline the documentation process.
And finally, thank you team- Prerita for the constant motivation to push through, even with all our other commitments, Arnav for helping us around Figma and making our documentation look much better and presentable, Nisheta for all the jam sessions and for bringing such great ideas to the table and Sivaprakash for being the go-to person for a second opinion and giving the best perspective to make us all think differently.
I can't thank you guys enough. So I am gonna take leave here. Until the next mission, should you choose to accept….Just kidding!😅
Secret agent Venkatlakshmi?🤔
🕵🏽♀️I am a secret design spy, who you can very easily find on LinkedIn, Instagram, Twitter or email. Hit me up for some juicy design details and general life advice!
