Project Members: Merrin Overbeck, Shauna Morrison, and Mikaela Weiss
Introduction to the Project
When we did the first assignment for this class, we found that many of our edits were quickly reverted by more experienced users without a thorough explanation. We decided that for this project we would investigate why this was and what we could do to improve the Wikipedia experience for new users.
We determined that the best way of measuring this issue is through investigating the retention rate of beginning users on Wikipedia. Retention rate, as defined by the Wikipedia retention rate page, is “the ratio of the number of retained customers to the number at risk.”
Through reading various studies, we found that new users are entering an environment that is more challenging and critical, contributing to the lower retention rate.
Importance of Project
It is important to resolve this problem because Wikipedia is an open collaboration system that requires a pool of volunteer contributors to function. Since all participants in such systems will eventually stop contributing, a steady stream of newcomers must join the community in order to maintain productivity. We decided to do this project because these were problems that we all faced throughout the course of this class.
Retention rate and new user experience is an important topic area of Wikipedia because it determines the success of the site as a whole and whether or not it is able to fulfill its purpose of serving both “news” and “history” at the same time. Making sure that Wikipedia continues to exist is vital, according to wordyenglish.com, because “Wikipedia is of undeniable usability, in many cases and aspects matching or surpassing those of professional published commercial encyclopedias as well as specialized dictionaries, compendiums, annals, or the entire collection of FAQs.”
Another component of why this project is important is due to the fact that according to “Kids these days: the quality of new Wikipedia editors over time,” the majority, approximately 80%, of new editors are not out to harm Wikipedia.
The technique used in this project was a combination of conducting interviews with experts combined with our own experiences and research conducted regarding retention rates and new user’s experiences. The innovative component of this project is that we actively created suggestions that could be implemented into Wikipedia. We compiled these various components into a cohesive study about Wikipedia’s retention rate and what could be done to improve it.
As stated previously, we collaborated with users such as hosts of the Teahouse (which will be explained later). Our project took the initiative to address a topic that is very important to Wikipedia’s success through compiling interviews and research.
Summary of the Problem
Throughout our research, we found that Wikipedia has been suffering from a declining retention rate of users. This is especially detrimental to Wikipedia because the site relies on users to contribute content, and if users are frequently leaving, then Wikipedia is at risk of having no new content created.
This issue is especially important now because between 2004 and 2007, the rate of active editors joining was constantly increasing, despite the fact that the one year retention rate had been declining. Starting in January 2007, not only was the one year retention rate declining, so was the number of active editors. This means that before 2007, Wikipedia could make up for the fact that users were leaving after a year by replacing them, but now the number of users joining Wikipedia is also decreasing, making it difficult to replace the lost users. This is demonstrated in a chart below.
We found that 60% of new account creators never make a single edit after the first day and that 0.01% of editors contribute 44% of the encyclopedia’s value. While this rate has fluctuated greatly between 2001 and 2006, the fact that the majority of edits is coming from a small proportion of users is a major problem facing Wikipedia still. If there is only a few users making all of the edits, Wikipedia faces the challenge of replacing these users with equally productive users. Below is a graph of this trend.
Another important statistic that we found during our initial research was that in a survey of 1200 former editors conducted in 2010 found that 27% of former contributors cited the rudeness or stubbornness of other editors as their primary reason for leaving. In order to maintain the productivity of Wikipedia, we need to make sure that users are not rude or stubborn.
Another problem that we found during our research was a major gender disparity in recruitment. We found a study titled, “Tea and Sympathy: Crafting Positive New User Experience on Wikipedia” conducted by Jonathan T. Morgan, Siko Bouterse, Sarah Stitch, and Heather Walls. This study addresses information about the Teahouse and challenges that new users face. Below is a screenshot of the article.
While we learned about this issue in class, through doing this project we found that this problem of gender inequality on Wikipedia originated in the recruitment of female users. We found that new women editors tend to participate at a lower rate than their male counterparts and see their edits reverted at a higher rate, thus causing them to leave sooner. These female editors rated their satisfaction with editing lower than men on average, citing the negative social factors at a higher rate than male editors.
Another factor that contributed to the poor retention rate of Wikipedia is that there are missed opportunities for socialization, meaning that it is too hard for users to find social groups because most users user quit after day. When users are able to form bonds with other members of the Wikipedia community, they are more likely to stay active on the site because they will have people that will help them and share similar interests with them.
The final factor that contributed to the poor retention rate of Wikipedia is the isolation and intimidation that new editors face. Regarding this issue, we found that users are likely to not contribute through making edits because there is confusion about how to use the software, concern about being the target of aggression or hostility, and uncertainty about whether the Wikipedia community was a good fit for them. Another factor that contributes to this feeling is that new users tend to only interact with bots, rather than people, when they have done something wrong. We found that the proportion of new editors whose first interaction with Wikipedia is via a generic message left an automated script or tool has increased to over 80% from less than 40% in 2006.
Personal Story demonstrating the problem at the center of our project
When SubSubPop23 first joined Wikipedia she was excited about editing. The first article she creates is about her favorite San Francisco street artist, a local legend. She logged in a few days after starting her article to continue working, but she sees that it had been tagged for immediate removal from Wikipedia because of a “lack of notability” and “few reliable sources”. She felt confused, unfairly singled, and frustrated. But above all, she was concerned that she was about to lose all of her work. She tried to make sense of the process for contesting the decision, but she found that the process was overly complicated and that the whole experience feels impersonal.
Quotes from Now Inactive Users
Below are quotes that we found on the WikiProject Editor Retention/Discovered reasons given for leaving Wikipedia.
Some reasons that Wikipedia users have given for leaving are: “edit creep, failure to recognize edit creep, cranks, lack of adherence to or understanding of scholarly values, vandalism, procedures, and a cumulatively dysfunctional system”
Trends in beginning edits
In our research, we found a correlation between the decline in new active editors and the rise of warning issued to new users. We found a study conducted by the Wikimedia Foundation that asked the question of whether editors are being warned more aggressively than those who started using the site in 2001 to 2006. Their research found that approximately 80% of new editors are not out to harm Wikipedia. Approximately about 40% of new editors are leaving valuable contributions. Despite this, however, the rate of rejection of first contributions have been rising steadily and retention rates have been falling. The most common edits are editing an article, creating an article, and adding photos and other media. This means that when we are thinking about our suggestions, we should focus on helping new users achieve these tasks.
Through doing this project, we have two main goals of doing this project: researching how to create a higher retention rate for beginning Wikipedia users, as well as coming up with new ideas and changes to suggest to Wikipedia community leaders to make the community more welcoming.
What is currently being done to address this issue?
One of the main projects that was created on Wikipedia was currently the Teahouse, which is a page aimed at creating a friendly environment in Wikipedia. The Teahouse was built around two primary activities, guest pages and the question and answer board. On guest pages, new editors can introduce themselves by creating a simple profile and browse profiles created by other new editors and by Teahouse hosts. On the question and answer board, guests can ask, read, and answer questions. Based on further research into this topic, we found that new editors tend to enjoy their Teahouse experience because the promptness and quality of the answers they received, as well as the friendly atmosphere and the ease of use. The Teahouse is the most significant way that Wikipedia tries to retain users. Below is a screenshot of the homepage to the Teahouse and of examples of messages that users would receive from the Teahouse.
There are also community-created mentorship programs that exist, but these exist on a relatively small-scale. One of the most successful programs, Adopt-a-user, found that it had served approximately 1,000 new editors between 2006 and 2011…but during that same period more than 7,000 new users created an account. This means that while the program reaches a significant amount of people, the majority of people who joined did not become a part of this program. Below is a screenshot of the Adopt-a-user page.
We also found that there is a function called the “Help Desk” which is a tool that users with questions can go to and get there questions answered. Below is a screenshot of the Wikipedia Help Desk.
Finally, there are also edit-a-thons and other outreach events to attract more users. These events are useful in attracting more members because they demonstrate to potential users that there is a community behind Wikipedia, even though it might be hard to tell when users first join. Below is a picture of an edit-a-thon held at the Smithsonian Institute Archives.
Interview with Jim Heaphy
We found that in order to have the best and most current information about the Teahouse, we decided that we would interview Jim Heaphy, who is “one of the most active Teahouse hosts and role is to assist newer editors by answering questions about how to edit Wikipedia”. We wrote a series of questions to ask him and then posted these questions on his talk page.
Below are the questions and answers that we received from Jim Heaphy.
Q1: What is your role in the Teahouse?
A1: I am one of the most active Teahouse hosts and my role is to assist newer editors by answering questions about how edit Wikipedia.
Q2: How long have you been doing what you’re doing?
A2: I have been editing Wikipedia for almost eight years and began Teahouse work shortly after the Tea House project began.
Q3: What is currently being done to create a more welcoming community for new members?
A3: The Teahouse is just one of many things that welcomes new editors to Wikipedia. Welcome messages are placed on talk pages of new editors. We also have a Help Desk which can be used by any editor. We sponsor edit-a-thons and other outreach events. “Wikipedia:WikiProject Editor Retention” exists to promote and coordinate and encourage these types of efforts across Wikipedia.
Q4: How do you think that you can improve what is being done?
A4: I am always trying to improve the clarity and usefulness of the answers I give at the Teahouse and on my talk page. I edit Wikipedia in order to improve the encyclopedia, whether directly by creating and expanding articles, or indirectly by assisting less experienced users.
Q5: How effective do you think the Teahouse is in increasing user retention?
A5: I believe that the Teahouse is useful with editor retention but my main goal is improving the encyclopedia. Improving editor retention is an important element of that but many people want to try to write a single Wikipedia article and then they move on. If we can assist them to write one useful encyclopedia article, then the encyclopedia is improved even if that editor moves on to another hobby. We should not want to retain chronic unproductive editors unless their behavior can be transformed, which is rare in my experience. Any editor who strives to become a long term, experienced, productive editor should be given all of the friendly help that they need.
Q6: Does the Teahouse currently have any major projects that its working on?
A6: Just recently, the Teahouse shifted from top posting to bottom posting, and we are now discussing frequency of archiving. The Teahouse is not really a place for “projects”. It is a place to provide friendly, helpful answers to the questions that new editors have/
Q7: Do you think that the current system of just answering users’ questions is enough to generate a higher retention rate?
A7: The Teahouse exists specifically to answer editor’s questions and is not intended to be a broader editor retention project, as I see it. If the Teahouse contributes to that broader effort by doing its own job well, then that is another benefit.
Q8: How many welcome messages does the Teahouse send in a week?
A8: Jtmorgan is also the best person to explain the latest research about the Teahouse and editor retention.
Jtmorgan, who was referenced by Jim Heaphy, has conducted research about the Teahouse and editor retention. He stated that he sends approximately 200 invite per day sent to new users to the Teahouse. He stated that the Teahouse has a measurable, positive impact on new retention. Below is a screenshot of Jtmorgan’s comment during the Jim Heaphy interview.
The chart above is one of the first charts made after starting the Teahouse as a pilot program. It analyzed the amount of questions asked in comparison to what category they were in. The chart below discusses the number of new editors a week over time just in the beginning of the Teahouse program. Jtmorgan advised us to look at these charts and the research pages that have been done in the past and present on the Teahouse and its impact on the Wikipedia community.
In order to improve Wikipedia’s community and its retention rate, there are a variety of possible solutions.
The first way that the retention rate of Wikipedia can be improved is by creating more opportunities for positive socialization. In order to do this, Wikipedia should design improvements to promote more female contribution to Wikipedia. This can be done by creating forums to facilitate social or educational discussion separate from the hypercritical, debate-style discourse of talk pages. This would allow users, not just females, to have a more friendly place, outside of the Teahouse, to have a friendly, welcoming place. Wikipedia could also focus on creating features for determining potential common interests and fostering more direct collaboration. This would allow users to create more of a personal bond with other users on Wikipedia, which will make Wikipedia a more welcoming place.
Teahouse could also advertise who they are a little bit better as well. All three of us had not heard about who the Teahouse was or what they did, until we started to look at the user retention in this project. The marketing that Teahouse has done is not enough and if new users know that there is an environment that they can feel comfortable in then they are more likely to take advantage of this service, and be more likely to stay on Wikipedia.
Because this project is centered on using retention rate, there are a variety of challenges that arise due to the inherent nature of retention rates. Retention rate is difficult to measure because it is difficult to know people’s true identities. This is so because users create a username which makes it challenging to track user activity. Also, there is the possibility that users will switch the language version of Wikipedia that they are using, which creates the challenge of tracking user activity across the language version.
The users that look at beginning editors’ edits face the challenge of being friendly and nice versus ensuring quality of edits. This is the main reason and limitation that creating a nice environment has on the Wikimedia community. The quality of edits are what is trying to be protected when these editors are automatically deleting articles and fixing things. However, the Wikipedia community has gone from an environment that anyone can edit to an environment that is protective and elite. Wikipedia has lost their founding principle in trying to create top notch articles, which is not bad, but they have made it a “either or” situation. There is room for both in the wiki world, but there just needs to be a balance.
Finally the last major limitation that this project faced was having these recommendations reach a substantial portion of the Wikipedia community within the time allotted for this project. If there was more time to do this project, we would have conducted more research into who would be the best user to talk to in order to implement some of our suggestions. Addressing the fact that we are required to document our project on Wikipedia, we determined that the best way of doing this would be to post it on Jim Heaphy’s talk page. We decided to do this because he is the main user active on the Teahouse; therefore, he is the user that would be most likely to be able to implement the suggestions that we made. We provided the link to this blog post to him on his talk page.
Division of Labor
When doing this project, the first step of the project was to come up with a topic that met the project requirements. When we first started brainstorming, our first project proposal was too broad so we had to work together to come up with a more narrow topic regarding improving the enjoyability of the Wikipedia experience. After determining a topic, we all worked together to write up questions to send to Jim Heaphy. We also all conducted research regarding Wikipedia’s retention rate and compiled it into a cohesive presentation.
When we first started this project, our proposed topic was too broad. After narrowing it down, we were able to come up with a project with possible major implications for Wikipedia as a whole involving the new editor retention rate. Through examining a combination of personal experiences and research, we were able to come up with a variety of different suggestions that would be able to be implemented in the Wikipedia community in order to address this issue. We learned important facts such as that most beginner users are not out to harm the website, as well as the fact that the majority of Wikipedia’s content is produced by a very small portion of users. In order to take advantage of these two facts in implementing a better strategy to attract new users and to encourage existing ones to stay, Wikipedia needs to create a more active approach to improving its user retention. While the Teahouse has had positive impacts on user retention, there are still many things that Wikipedia could implement in order to improve its user retention. Through creating more opportunities for positive socialization, informing new members that they are not the only experiencing this criticism, creating a more organized beginner’s guide, and improving the marketing strategy of the Teahouse so that more users are aware of it, Wikipedia can make sure that all of its users have a happy and productive experience.