Public Review Case Study

UX Research, MVP Development, Strategic Roadmap Design

As a short story writer who has both submitted to and worked at literary journals I knew there was a problem with the system currently employed to filter submissions. Specifically, the sheer volume of submissions far outstripped each journal’s capacity to adequately review each piece, creating a massive backlog of work to be filtered. The top-down hierarchal system in place was strapped for resources. One day while listening to a podcast on the subject of the wisdom of the crowds a solution hit me, what if the submitters themselves could become a part of the process, what if they could review each other’s work?

The Wisdom of Crowds = the theory that a larger group of diverse people can make better decisions, and display more intelligence than any smaller collection of experts. http://p2pfoundation.net/Wisdom_of_Crowds

The first step in developing my product idea was to get out of the building and start talking with potential users to see if they shared the same pain.

User Research

What I discovered during my interviews was that although many writers considered the current process frustrating, one called it, “rubbish,” another described it as, “maddening and unfair,” their biggest pain point was not knowing whether their work would actually be read. In fact, many of the writers had basic to no experience with standard industry editorial practices, which upended one of my core assumptions. However, every single writer I interviewed made a point to emphasize their desire that their work would be read by someone else, which is not guaranteed by the current system. This insight led to a deeper understanding of the problem at hand and an expansion of the overall vision for the product’s value proposition.

Frustration With Current Process
80%
Desire to Have Work Read
100%
Desire for Compensation
70%
Writers
0
Interviewed

Key User Persona – Tim

Age: 24
Education: BA – Liberal Arts School
Location: NYC
Work: Boring day job

Tim is currently writing and submitting work for publication but is discouraged by a process that takes at least 6 months to return a stock rejection if he is lucky.

Tim understands this as he is currently working as an unpaid intern at a magazine where he reads through the slush pile of submissions and sends out stock rejection letters to the countless writers.

User Insights

Armed with the insight gained from interviewing prospect users I refined my value proposition as well as the core user pain that my product was meant to solve: guaranteeing that their work would be read. This is what led them to participate in writings groups, submit their stories to journals, and was the core motivating force in the creation of their work. Their key frustration came from the lack of feedback inherent in many of these processes. Further, this frustration is not confined to literary publishing. It is also shared in any process that requires an exceptional amount of work and results in little to no feedback. If The Public Review model could work for literary publishing it could potentially be applied to much larger industries involving large sets of submissions with limited resources to review such as:

  • Job Applications
  • Contests
  • New music discovery
  • Long-form writing
The Public Review is a quarterly online literary magazine powered by a web app which empowers its writers/users to review each other’s work for potential publication while guaranteeing that each submission will be read. The Public Review's Value Proposition

With the knowledge that a larger market existed should the model be proven in this niche, I set out to deepen my understanding of The Public Review’s users by running a series of landing page A/B testing to refine the core value proposition of the MVP for The Public Review.

A/B Testing

Process & Results

I setup a google adwords campaign to drive traffic into an A/B test launched on unbounce.com. The initial results weren’t promising, as the page converted 0% of traffic into e-mails using the initial generic landing page. Continuing to experiment using data from the user interviews to elaborate on the product vision I was able to gain traction with the message “Calling All Writers” and a simple breakdown of the product’s value proposition. After two days I had sufficient data from my adwords campaign to clearly show that the call to action “Submit Your Story” significantly outperformed the more passive “Calling All Writers.” Further, applying this change to the landing page resulted in a 34% increase of conversions with a final result of 12.74% of traffic converting into e-mails. With the value proposition validated I set out to develop an MVP prototype with the core features needed to test the viability of The Public Review.

Key Insight: The “Submit Your Story” call to action increased conversions by 34% over the generic phrase “Calling All Writers”

Generic Landing Page – 0% Conversion

Clear Call To Action – 12.74% Conversion Rate

Web App Prototype MVP

Using the data gained from the live A/B tests, Google Adwords campaign and user interviews I set out to build a prototype that contained the key features needed to test the viability of The Public Review as a product. At the core were the following needs:

  • Your story will be read. It is important to emphasize that each submitter will have their story read by other writers.
  • Every user must read and rank stories in order for their own to be considered for publication. This is the key carrot-and-stick to drive users to review work and power the app.
  • Anonymity – the process of reviewing and ranking needed to be anonymous. To make the ranking accurate each reviewer needed to work in a vacuum. One interviewee said, “I wish more places wanted their submissions blind… someone’s bio can easily trump their content.”
  • Credit and Profit. One interviewee said “credit is always good. I tend to take things more seriously if my name is attached to it.” Each writer who finishes the review process will be credited as an editor upon publication. Those who have their stories are published will receive a share of the profits. This connects them as participants as well as future readers.

With these features in mind I set out to develop a prototype for The Public Review web app, which can be seen in the wireframes below:

Home Page

Account Creation Page

Story Submission Page

Story Ranking Page

Reading Page

Thank You for Editing Page

Purchase Issue Page

Customer Channels

  • Online – PR outreach to literary publications & blogs, continuing search ads
  • Social Media
  • Kickstarter campagin
  • Eventually users will be able to share 3 copies of the publication they paricipate on

Metrics For Success

  • Users who submit / users who complete their reading and rankin assignment during one publication window (if users don’t rank stories the app will not work)
  • The overall quality of the output (if the system outputs poor quality stories then it may not work).

Roadmap

0-3 Months

  • Connector.

    MVP Development

    Build out an online web app with only the core features needed (initially may not even have a sorting algorithm, all initial functions can be done by hand without user’s knowledge).

  • Connector.

    Initial MVP Test

    Run simple test of the idea using a creative writing class to get immediate feedback on the key features: anonymity, viability of the process.

3-6 Months

  • Connector.

    First Cohort Test

    Using insight from the initial classroom test run a live test with real users of the MVP web app.

  • Connector.

    Continue to Refine Product

    Track key success metrics to evaluate viability of the product and iterate on its value proposition.

  • Connector.

    Add Back-end to MVP

    Based on initial in-class testing and results from the live user testing of the MVP, build out the backend of the key features.

6-9 Months

  • Connector.

    2nd Cohort

    Run second cohort of MVP continuing to test new features and reliability of the app.

  • Connector.

    Distribute First Issue

    Sell digital copies of the first issue (with results from the first cohort). Sell print-on-demand physical copies. Give 3 free copies to each participant to distribute and fuel user and reader growth.

  • Connector.

    Continue Refining Product

    Track key success metrics to evaluate viability of the product and iterate on its value proposition.

9-12 Months

  • Connector.

    Run 3rd Cohort

    Continue refining product and features.

  • Connector.

    Create MVP Product Extensions

    Create MVP’s for music, applicant tracking systems, and contests.

  • Connector.

    Test New MVP's

    Test the new MVP’s to discover viability and iterate their value.

1 year+

  • Connector.

    Pivot Brand

    Pivot the brand into the larger market. Drop the “The” and become: Public Review. (The Public Review will continue to operate as a quarterly literary journal and decouple from the app itself, which will be Public Review.)

  • Connector.

    Launch and Test New MVP's

    Launch, test, and get feedback on the MVP’s for larger products: music, applicant tracking, and contests.

  • Connector.

    License the Content and Technology

    Approach publishers to considering license the Public Review technology, as well as the content published.