How can an online community measure its growth and development?

Question

Is there a more effective way to measure how a community is growing and developing than just with simple metrics like:

• Total membership
• Number of new members per week or month
• Number and percentage of members returning in a period

Answer 1

The online and digital components of a community provide many opportunities to measure what's happening between the members of the community. There are so many ways to measure this activity, its actually possible to drown in metrics and not necessarily derive any clear meaning.

Measuring activity in a community, however, can provide a lot of insights of how the community is running, what is interesting and motivating for members, and what might be blocking efficiency of the community. In addition, when the metrics are shared with members, it can provide a call to action and bring inspiration for the groups purpose.

Measure goals related to your purpose.

Usually any community exists for a purpose or mission. When organizing the community and making strategic plans, say on an annual basis, there should be some goal statements related to that purpose or mission along with measurements to show how the community is progressing towards these goals and the mission. For example, perhaps the community exists to help bring a certain level of experience to its constituency. So the goal might be "X number of people pass beginner and advanced certification tests this year".

This topline number is the overall performance of your community.

Map the maturity cycle of your members.

Next, its useful to map the journey of what a member would take from first learning about the community to becoming "counted" in relation to this overall goal. A new member would first become aware of the existence of the community, then would choose to join and engage in some manner, and finally reach an "achieved" level. This achievement level might be the end of the journey, or it might simply be the point where the member is able to contribute effectively.

This journey can then be thought of as a funnel, not unlike a marketing or sales funnel.

Aware -> Engaged -> Achieved

Already you can see an opportunity for more measurements here:

1. Number of members Aware
2. Number of members Engaged
3. Number of members moving from Aware to Engaged
4. Number of members Achieved
5. Number of members moving from Engaged to Achieved

Now map all the ways a member participates when they are in these various stages of their journey.

Measuring awareness.

Awareness usually happens on the fringes of the community. Examples are websites, social media, and third party sites. Here web statistics are quite useful, such as what you can get from Google Analytics. Number of hits, users, new users, returning users, and referral sources

If your journey map is detailed enough to know the path an aware member would take towards becoming engaged, its possible to measure something like # of sessions moving on engagement path, and percentage of sessions ending with engaged member.

I would also count physical attendance at events in this category. # of people attending an open event, number of new attendees.

Measuring engagement.

Part of mapping your journey is knowing what a member needs to do to become "engaged". This could be as simple as creating a profile, entering contact information, subscribing to a newsgroup, or as high a bar as sending in membership dues. The number of people completing this "call to action" is the base number here.

Once these members are engaged, you need to measure whether they stay engaged, and how close they come to making the next step of the journey to becoming engaged. I would be looking at metrics like:

1. Number of members participating in communications
2. Attendance at events
3. Number of newly engaged members returning daily or weekly for a period of time.

For my own communities that are dedicated to collaboratively building open source software, we measure developers engaging in the collaboration tools, as well as activity in our mail list. In fact, we post this information publicly so all the members can see: http://projects.bitergia.com/apache-geode/browser/

Try to figure out a common sequence where newly engaged members are more likely to stay engaged, or achieve the next step in the journey. For example: when newly engaged members have an assigned mentor, and also attend 6 meetings in a row. Then try to funnel newly engaged members into a program around such a pathway.

Measuring achievement.

Similar to becoming engaged, you need to know the specific actions as to when you member can be considered reaching the "achieved" level. This might be passing a test or demonstrating a certain kind of behavior. It should be related to the mission and goals of your community.

In addition to measuring # of members moving from "engaged" to "achieved", you will also want to measure "Time to becoming achieved". This latter metric actually tells you the efficiency at which your community is able to create full fledged members, and achieve its mission.

You will want to measure the leading behaviors that bring a member to the achievement level, such as if they are taking courses, or accomplishing a number of sub tasks along the way.

For example, in the case of our open source software communities, we track number of downloads, usage of tutorial information indicating self-teaching, and the point at which the user can prove a beginning level of competence with the software. We call our metric "Time to hello world".

Of course you would continue to measure engagement of your members at the "achieved" level. Assuming there is more for them to do, you should generally expect a higher level of engagement among these core members.

Measuring contributors and volunteers.

Finally, another important metric is measuring development of super users, community elites, or leading members. You will also want to map this path. Is it merely the next step after becoming "achieved", or is it a separate pathway? For example, if the purpose of your community is to teach a skill, someone who is already an expert by other means may have a direct path to becoming a super user than coming from the engaged & achieved route. In a nonprofit community, these might be your volunteers or staff.

They key thing to understand about these super members or leading members is that you want them to contribute to the good of the community, its purpose, and for the benefit of engaged and achieved members. The top line measurement of this would be "time to first contribution". Of course you will also measure engagement of these members, but its important to understand that they participate for their own kind of reason that may or may not be directly related to the purpose of the community. Fortunately they tend to be a smaller number, so its easier to cater to their qualitative differences. This group, however, will often be your biggest donors, biggest promoters to drive awareness, the reason why many members will want to become engaged, and are a potential source for leadership for the community.