The story of how we listened to our users and made a better product

Everyone in the tech industry knows that a deep understanding of the users’ needs is the foundation for building a product that users love. Although it has still to be proved if Ludwig is something that users love, my team and I tried to listen and understand our users at our best. As far as we have gone in this adventure we learned that there are three ways for knowing what users want:

  1. Being a visionary genius who can neglect customer feedback and still build the perfect product simply based on his/her prophetic gut feeling
  2. Figuring it out from how users interact with what you built
  3. Talking to users and letting them tell you what they want
    I’ll dig a bit deeper into each of these three approaches.

1. The visionary-genius way

Unfortunately, this is not our case. We are not geniuses, just fairly smart and hard workers. This means that we had to figure out what our users wanted the hard way, i.e. by talking to them and inferring it from their behavior. If you are a genius: congrats! You can stop reading here.

The figure-out-how-users-interact-with-your-product way

It is no secret that apps and websites try to figure out how users behave and interact with their products. Understanding what pages are most seen, in which order, where users click is a great way of understanding how you can build a product that suits your users’ needs best. On the side of honest and fair ways of collecting and using users’ data (I’ll call it the Gandalf way), there are also less transparent – I’d say shady – ways of profiling users’ behavior (I’ll call this the Saruman way). Facebook’s datagate is an example of how profiling can be filthy and hateful and a huge red flag on how our data is treated.

Mark (Saruman) Zukenberg while collecting users' data

Actually, you don’t need to have a very complicated tracking system, nor pay for expensive software, nor get sensible data on your users that at some point may blow up in your face (this, by the way, is our privacy policy). Our goal was not to set up the perfect click-counting machine that kept track of everything happened on our website but simply make sense of users’ behavior. We keep track of few metrics so that we don’t get overwhelmed and I personally prefer the simplest ones (i.e. with fewer variables).
Our stack for understanding users’ behavior and our metrics is currently made of:

  • Google Analytics: Almost every single website owner in the globe uses Google Analytics to track their users’ behavior, so do we. Data are completely safe and anonymized and you have plenty of information to mine. It took us a while to set it up correctly to track the specific events that you want to keep track of (e.g. how many Italian users clicked on the dictionary function today). We don’t track too many metrics. We set a number of conversions that are reached when a user is registered, buys a subscription or downloads our app. The thing I like the least about Google Analytics is Google knowing too much our business metrics. My team and I discuss this a lot and will probably develop an in-house solution.
  • Kibana: This is the data visualization plugin for Elasticsearch (a search engine based on Lucene). We use Kibana for a lot of things. It is currently one of the best candidate to free us from Google Analytics. We use it to monitor several metrics and to make sense of Ludwig searches and infer if the response from Ludwig was satisfactory for the user.
  • Stripe: Stripe is the web-standard for managing payments and subscriptions. They take a huge commission on every transaction but, given that is very safe and reliable, it’s worth the cost of not having to handle credit cards and payments. From Stripe we can keep track of who is paying, how much and from where.
  • Sendinblue: We use it to keep track of the email marketing metrics, see how many subscribers open our emails, how many readers click on the links and how many reply back. Since we personally hate receiving useless emails and weekly newsletter and pushy updates from the services we subscribe to, coherently we decided that we would not abuse of our users’ patience and of their email boxes. We email our users only when we have to communicate something compelling (if you exclude the technical emails for registration and password recovery), which happens very infrequently. Contrarily to what 99% email marketers think, we believe that the stereotypical weekly newsletter, theoretically made to create engagement, actually repels readers (or at least would repel me as a reader), customers and engagement. In fact, in our case, not being pushy paid off and our users appreciate it. Our opening rates (approximately 40%) and high clicks (7%) double the industry average, plus we have a lot of people answering. This last aspect brings us to the next point: talk to your users.

We feed the information on behavior and metrics we extract from the above mentioned tools to our funnel (if you don't know what a funnel is take a look at this post from 500 startups in which we look the customer journey of groups of users with our product.

Talking to our users

We are just so lucky that we didn’t have to put a lot of effort into it. Possibly because we don’t write to them often, they write spontaneously to us. Our users love to talk to us and we love to talk to them. Roberta (my co-founder) and I personally replied to more than 6000 feedback emails each. Yeah, that’s a lot of feedback that has to be dealt with. Sometimes it’s overwhelming and time-consuming but also rewarding. During the last 4 years I even established personal friendship bonds with some of our users. In our case this kind of feedback spans from “I am a fashion blogger, why don’t you add Vanity Fair and Vogue as a language sources?” to “How long should I wait to have Ludwig for Android. Please make it as soon as you can!” and includes “Make Ludwig for free, plz!”, and “Can I use it to translate from English to French?”.
As a rule-of-thumb, you don’t take all of your users’ feedback literally. There is a well-known quote allegedly attributed to the American inventor and entrepreneur Henry Ford, founder of the Ford Motor Company, that says:

“If I had asked people what they wanted, they would have said faster horses.”

Users perfectly know what they want, though they might not always be able to articulate it with precision. Keeping this in mind is key and the only way to circumvent the problem is by asking specific questions. For example, when a few users told us they were using Ludwig on one tab of their browser on the side of their writing app, we figured out that we needed to build a desktop app so that Ludwig is more seamlessly integrated in the writing workflow.
If you ask the right questions, you’ll get excellent information that will help you glean new insights on how to evolve your product.

The secret ingredient

There is no secret ingredient. It’s about doing the same things over and over and improving the process. A/B testing is a great way to move fast and get better day by day. We do some A/B testing, but we could definitely do more. Good luck with it.