Is startup validation bullshit?

Shortly before starting my software app Informly I read the Lean Startup and had lots of conversations with some very smart software entrepreneurs about validation. I also read lots of posts from people who had ‘validated’ their ideas and had great results.

The problem is what happens if your results aren’t as rosy as those examples?

I feel I can offer a unique perspective here because I launched a product (the first version of inform.ly) last year that ultimately failed. It had over 4,000 free signups and only 15 paid customers ($9 / month). I eventually shut down that version of the product. Here are some of the things I learnt.

(The idea was a dashboard that brought together all of your stats into the one place in a simple way).

Validation doesn’t work so well when the answer isn’t an obvious YES

The popular stories you hear about startup validation go something like this:

  • I created a website with a brief video.
  • It went viral.
  • I bought a yacht.

Dropbox started from a 3 minute video posted to Hacker News (more here). As they progressed through the beta they had their signup list jump to 75,000 people in 1 day. Here’s a cool slideshare from their founder Drew Houston.

Well that’s great if you can get 75,000 optins in 1 day sounds like a good idea but if you don’t is it a bad idea? Maybe you just don’t have much kudos on Hacker News?

The reality is that most ideas aren’t going to go viral. Let’s face it, the chances of you coming up with the next dropbox are fairly minimal. But even if your idea is great will a video showing it’s features go viral? There are a lot of factors at play there I wouldn’t be betting on it.

Email optin / beta signup totals don’t work

As with dropbox above and numerous other examples, email signups is often considered the key indicator of whether an idea is good. With the first version of Informly I had:

  • 1,000 people sign up for the beta (3 months)
  • 1,200 people entered their email to be notified of launch (around 7% but that was to the entire site and I was doing a lot of content)

These may seem like small numbers but for someone who had taken 5 years to built up an email list of 2,000 people in my last business I really felt like I was onto a winner.

However as I’ve found with this idea and other versions (more later) there is a very big difference between someone entering their email into a box and someone paying you each month for a product.

People saying it’s a good idea doesn’t mean it is

As part of validating Buffer Joel Gascoigne “simply tweeted the link and asked people what they thought of the idea”His post on validating Buffer indicates that he read a lot into this including:

“After a few people used it to give me their email and I got some useful feedback via email and Twitter, I considered it validated. In the words of Eric Ries, I had my first validated learning about customers”.

Really? A few of your mates said it was a good idea and therefore it’s ‘validated’? Obviously it turns out that Buffer was a good idea and a product that people wanted and were willing to pay for but does that mean the validation technique was a good one? I don’t think so.

Here is one of the most well known people in the startup space saying that my idea for Informly was great:

Here are some of the things that people around me said at the time of pulling it all together:

  • “I’m not sure if this email will make it to you, but you’ve managed to build the software most of us wished we already did!” (didn’t end up becoming a paid customer)
  • “Thanks for helping to solve a problem most of us face every day” & “Great work man! I use this product frequently and have recommended it to quite a few people.” (didn’t end up becoming a paid customer)
  • “I have just jumped onto the new platform from http://inform.ly and love it!” (didn’t end up becoming a paid customer)
  • “I’m in love with it. Let me know if you ever need a testimonial. I’ve been waiting for this all my life”. (didn’t end up becoming a paid customer)
  • “Hey Dan! Informly is amazing! What an epic idea. You can manage everything that matters from one place” (didn’t end up becoming a paid customer)

I could go on. I haven’t included any solicited testimonials above, I had lot of friends telling me it was a great idea and giving me great testimonials as well. But those above are just the unsolicited ones.

Coverage in tech press doesn’t work

When I launched the first version of Informly I had it down as a validation goal to get featured by a respected international tech publication. I would have definitely settled for a local Australian one. In the end I was well stoked to get covered by these and more:



This was pretty remarkable given:

  • I’d never built a software product before
  • I had more or less $0 marketing budget
  • I had no network in the startup community
  • I had no funding and no co-founders

According to this post by Noah Kagan I should have at least got 12,000 users from the Mashable coverage. Combine that with the others and I should be onto a winner right? Guess how many paid users were sent by the above traffic sources? Zero.

Targeted Surveys don’t work

content_analyticsNot long after realizing that the idea wasn’t going to fly I decided to ‘pivot’ to a content marketing Analytics app. This time I was determined to make sure I ‘validated’ the idea before working on it for 6 months.

So I built a targeted list of keen content marketers and asked them a bunch of questions in a survey. The purpose of the survey was to ascertain:

  • Were people measuring their content marketing
  • Would they pay for a tool that enabled them to measure it.

It even included specific questions like ‘Would you pay for this if I built it’. The results are above and they are clear (or at least they were to me at the time):

  • People generally weren’t measuring the important metrics for working out if their content was actually resulting in more business.
  • People would pay for it. 20% definitely, almost 60% possibly. Going from a free to paid conversion rate of .3% on the first version of the product I was very excited about these figures.

After I launched however I found things to be very different:

  • Most of the people in the beta list didn’t even use the product.
  • None paid for it.
  • After launching it to the public the signup rate was well below the last version of the product (based on on-page conversions).
  • I had 3 people sign up to pay for it. 1 cancelled within a week, the other 2 weren’t actually using it yet.

I should blame the product for some of this as it was the best I could do within 6 weeks and needed improvement but the results were shocking to me at the time. How could they be so different from the survey results?

Well as Steve Jobs said ““people don’t know what they want until you show it to them.” The opposite is also true, people don’t know what they don’t want until they see it. Which brings me to the MVP.

MVP’s are a lot harder than they sound

Ah the much loved ‘MVP’ or minimum viable product. The concept, introduced by Eric Reis with the Lean Startup movement is simple. Do the least amount possible to ‘validate’ the idea. Build a ‘viable’ version of the product and get people using it as opposed to getting it perfect before launching.

I find it quite funny that Eric Reis talks about the startup Path in the book. A company who spend 6 months with their team behind close doors before launching path 2.0. The reality is that a lot of great designers and creators don’t want to release a crappy unfinished product. While it’s appealing to take the ‘vision’ component out of being an entrepreneur it often doesn’t play out that way.

Most ideas won’t sell as half-finished unpolished versions of what they should be. Most of the SAAS entrepreneurs I know are slaving away for months if not years on their idea with very little traction. Of course responding to customer feedback is important but is it even possible to release a week old product that people pay for?

It might be but it’s certainly a lot harder than it sounds. Services are infinitely easier. A week ago I launched a new service WP Curve, within 1 week it had built up more recurring revenue than a year working on Informly.

Why?

There are a lot of reasons but one is that I could easily release a full version of the product from the customer’s point of view within days. On Saturday I decided to launch a 24 hour developer service for WordPress support and small fixes. By Wednesday I had a website and I have the ability to deliver the service in full (for the first week it was me with a live chat on my mobile phone).

I don’t care what the lean startup gurus say, that is not possible with software.

Some things just take time

Nathan Barry

Nathan Barry is a well-connected, skillful entrepreneur who recently tried to build a recurring software business in 6 months. The goal was to get to $5,000 a month. He posted recently after the 6 months was up see this great post.

I recently saw Tim Ferris tweet one of Nathan’s blog posts. He is a very well respected entrepreneur, well connected and very skilled. Checkout the ConvertKit homepage <— legit.

It’s not clear to me how much recurring revenue he actually built since he mentions that he is including some trial people in his calculations. But its’ clearly well below half of the $5,000 goal.

Does that mean it’s a failure? Even if it’s $1,000 it’s more than me and more than most of the other software entrepreneurs I chat with regularly. Building a recurring software business is not easy and some things just take time.

Stop fucking with the product

One of the decisions I made in the last few weeks was to focus Informly back on it’s agency version (which sends simple reports to the clients of web design agencies).

To my delight there is a lean startup term specifically for this decision – The zoom in pivot! In reality though the decision really was to just take a breath and wait.

The irony is that this was the original reason for building the product and this version of the product has existed (in various stages of quality) for a long time – at least 3 years). If I wasn’t so obsessed to find a magical quick fix for the last 12 months perhaps I would have already turned this idea into a valid business.

For the first year I was desperate to find that single thing that took my idea from a dream into the next big thing startup. It didn’t happen. So I chopped and changed a thousand times to try to work out the right ‘formula’. Sometimes though the answer isn’t so obvious. The idea doesn’t go viral. It just takes time.

I’m getting regular new trial customers, a few new paid customers a month (more than the amount that are leaving). It’s no viral startup but it’s growing and over time might become something good or even great. Maybe it just needs a bit of space. And a bit of time.

About

Dan Norris is a co-founder at WP Curve and a passionate entrepreneur with an obsession for content marketing.

  • Micky Deming

    Great post Dan! This is a good conversation starter, but a few things I disagree with.

    1. I don’t think the concept of validating a startup necessarily points to “going viral.” The idea is to eliminate waste. To find out if what you’re doing works without spending a ton of time and money building something nobody wants.

    If anything, it’s less likely to go viral, because you are building from the bottom-up, one person at a time.

    2. You are right that people telling you an idea is good should not be considered validation. But I think the lean startup method would agree with that.

    There’s a big difference between saying you like an idea, and agreeing you will pay for it. But I think true validation is getting paid for something.

    I do agree, however, that certain software startups are very difficult to validate directly with minimal developing. I have worked with some companies who have spent a lot of time in this tension.

    But I do agree with Ries when he says that for entrepreneurs, whatever the minimum viable product is in your mind, the actual MVP is probably half that.

    • http://inform.ly/ Dan Norris

      Hey Micky thanks for the comment mate. The point I guess is if the validation is a clear yes or a clear no then some of these techniques will work but if it’s in the middle then they won’t. In my experience people saying they like the idea and people saying they will pay for it are equally useless. It was only when I asked for money that I really knew and no one would have paid for a half finished product.

      I think one approach that could work is pre-selling. Dane Maxwell and Clay Collins talk about this. The issue here is you need to be a good salesman or have a big audience. But I think it forces you down the path of validating ideas before building anything which is probably a good thing.

      • Micky Deming

        Completely agree Dan. Talk is cheap.
        Really appreciate the work you’re doing and the content you put out on a regular basis. Keep it up!

      • http://smartsoftwaremarketing.co.uk/ Giles Farrow

        Dan, I think Micky’s points are valid. Talking to people about your idea and putting up a signup page does not mean you’re doing customer development.

        Qualitative research does work — and you don’t need sales skills – in fact selling gets in the way. No-one will tell you that your baby is ugly so don’t test your idea for a solution until after you really understand if they have a problem they are willing to pay for.
        - Ask what they do now — and why
        - See if they have tried to fix it already
        - What is the impact of this problem / why do they care
        - Keep asking why
        If you hear passion that is validation. If you hear polite interest that means no.

        • http://inform.ly/ Dan Norris

          Good points. But having those conversations is a skill. Whether you call it sales or something else.

  • http://www.beachheadmarketing.com/ Steven Moody

    Agree about the service business, its far easier to start a business idea around a service and then automate it later. Easy to say in hindsight but informly should have started as a monthly dashboard service run by people instead of code. Cardmunch is a great example here: sold company to Linkedin that ran on Mechanical Turk.

    • http://inform.ly/ Dan Norris

      Probably true mate. Thanks for the comment.

  • Barry O’Kane

    Thank you.

    This is my favourite post in a long time. I get so tired of (what I see as) blatant survivor bias in so much start up writing. Your candidness is encouraging and inspiring.

    • http://inform.ly/ Dan Norris

      Thanks Barry I agree with you. I wanted to talk about that specifically but it would have been a bit off topic. If I hear one more expert tell me that they achieved success by ‘working hard’ I’m going to lose my shit.

  • http://www.wantrepreneur.com.au/ Alex McClafferty

    Hi Dan – your honesty is refreshing. I’ve witnessed the same phenomenon where friends and connections will tell you how good your idea is, but when it’s time to put up or shut up, they choose the latter. I know you’re a one man band – do you have any thoughts on partnering up with a more established entrepreneur in the future? PS – I think there’s a book in this … ‘Starting Up – I Failed But Still Won’. I’m a writer and would love to help write it!

    • http://inform.ly/ Dan Norris

      Hey Alex I’ve been in business for myself for 7 years so I don’t think I need more experience. What I do need though is someone with complimentary skills. If I could find someone who could do sales and customer development stuff and I could just focus on product and content I’d go for it. The problem is finding someone suitable who shares the same ideals, work ethic, someone who I work well with etc. I don’t think it’s an easy thing to do. If I was to do it again I’d consider doing it that way although I am a bit obsessed with getting stuff done myself (probably to my detriment).

      • http://www.wpcurve.com/ Alex McClafferty

        Lucky comment #100. ABC!

        • http://wpcurve.com/ Dan Norris

          *moderates comment ha

  • http://cescvilanova.com/ Cesc Vilanova

    Thanks Dan. This is probably the more real life post about validating business ideas I’ve read.

    Very relevant to what I’m “suffering” right now.

    This is what I did in my first startup attempt:

    Idea -> Build -> Launch day -> 1.000$ monthly revenue for the whole 2012 -> Unable to grow -> Shut down.

    This is what I did after reading Lean Startup by Eric Ries, Running Lean by Ash Maurya, etc.:

    Idea -> MVP -> 2 months testing manually a service with 20 users -> Not enough engagement (I considered) -> Discard the idea not knowing if the MVP was too simple.

    This is what I’m trying to do now, after following Steve Blank Udacity course “How to build a startup” (https://www.udacity.com/course/ep245) and reading more about customer discovery.

    Idea -> Forget about idea and try to focus on the problem -> Try to find people to talk about their problems -> Lot of time, difficult to get insights -> Never feeling the idea is worth pursuing.

    • http://inform.ly/ Dan Norris

      Probably wise but I don’t think I’m capable of it. Let me know how it goes for you. Thanks for the comment too.

  • http://multimediamarketingshow.com/ Jake Hower

    I’ve got nothing to add… I enjoyed the read though :)

    • http://inform.ly/ Dan Norris

      Ha thanks mate. You are taking a slightly different route with http://fuzedapp.com I’m keen to see how it works out for you.

  • http://www.bizsugar.com/ Heather Stone

    Dan,
    You wrote:
    “However as I’ve found with this idea and other versions (more later)
    there is a very big difference between someone entering their email into
    a box and someone paying you each month for a product.”

    Amen! Different scenarios outside the software field are no different. Advisers suggest you conduct a survey of potential customers in your market or create focus groups to determine whether customers are likely to respond positively to your brand. The trouble is that, in the end, the best way to figure out whether someone will buy your product or service is to put it on the market. Not to say there is no value in these other exercises. But it’s important to understand their limitations. Thanks for sharing with the BizSugar community.

    • http://inform.ly/ Dan Norris

      Thanks heather yeah I think it applies to all industries just selling an unfinished software product is not easy. It’s much easier to make it up as you go with services.

  • ryanwalker

    Responding to your provocative headline…

    Is “startup validation” bullshit? No.

    Validation is a journey from “I wonder if this is a good idea” to “Sweet, we have a profitable company.”

    If your idea doesn’t suck, you’re going to get early, positive feedback — which can mislead you into thinking the idea is “validated”… and off you go to build it (especially if you’re more developer than business person — because building shit is fun.)

    My favorite advice to startups is to ask people for pre-orders (with a specific price offer) on the product idea before building anything. Become a push salesperson. People will shift into “buyer” mode — and REAL objections will come out.

    Now you have something to work with — you need to hear criticism on your plan to solve the customer problem. Too many startups build a product before getting these objections, you’ve left a lot of risk in — and disappointment is more likely.

    Next step, build a MVP and try and collect revenue ASAP — launch super early and ask for money — expect that your product will be fucked up in enough ways that nobody will pay you. Then, keep working on it until you can get people to pay you. Or eventually give up.

    This is all incredibly messy (like life) and there are so many variables that following Lean Startup and Running Lean principles absolutely don’t guarantee success. Even if you get to the “sweet, we have a profitable company” graduation event from startup to company, your victory may be temporary :)

    I like your insight that it’s easier to start a service company than a product company. I encourage first-time entrepreneurs to build a company that wraps a small product with services vs. doing a pure product as their first entrepreneurial venture.

    Thanks for your post Dan.

    • http://inform.ly/ Dan Norris

      Thanks Ryan. I agree with everything you said. The only thing I’d say about pre-selling though is I suck at sales and I know people who don’t who could pre-sell anything.

      Does that make their idea more valid? I guess I was hoping I could find a sweet spot where I could focus in building a product but in the end if you are in business it’s hard to avoid selling if you want people to buy from you.

      • ryanwalker

        Good point — I forgot to mention that pre-selling must be done by the founders — not hired guns — because the goal is as much to learn from the objections vs. actually get an order (though getting the order is progress.)

        One other insight I HAVE to share… Recently we build http://cladwell.com in 11 weeks, and our team made more than 800 commits to the git repo during that time. I gave up on several solo projects after 100 or 200 commits, but were made over a much longer timespan.The insight is that being a solo founder adds risk. I’m pissed at myself for giving up on some of those projects so early (from a commit standpoint).

        I’m curious how many commits you’ve made to Informly?

        • http://inform.ly/ Dan Norris

          Wow that looks interesting I signed up. As for commits I don’t measure it but we’ve changed the product a LOT. Way too much. I’ve spent a year full time working on it for most of that year I had 2 developers working on it. Probably 70-80% of that work is not being used. It was either a different version of the product that I discontinued or it was a feature that we ended up removing. It’s been messy.

        • http://multimediamarketingshow.com/ Jake Hower

          Haha, I LOVE the endorsements on the homepage, they are gold!

          • http://inform.ly/ Dan Norris

            I tried the service last night it was great. I would totally use it except it doesn’t really work that well in Australia with the shipping costs etc. The process was great though and I’m the perfect market for that idea. I hate shopping, don’t know anything about fashion, don’t know anything about where to buy stuff online but I’m happy to pay for a service that tells me what to buy.

  • http://casjam.com/ Brian

    Great post Dan. Our startup community has a tendency to ignore all of the exceptions to the “rules”. But there are lots of them, and your businesses are perfect examples.

    • http://inform.ly/ Dan Norris

      Thanks Brian I’m glad you found it useful. We still need to do that podcast!

  • http://zapier.com Wade Foster

    Great post, Dan. I think people sometimes get caught up in iterating on the lean startup with these magical steps thinking that one day they’ll wake up with product/market fit. Truth is it’s a lot harder than that and a lot of products simply aren’t useful.

    Then when you find one that is, most products don’t take off like Dropbox. Instead they are more like Constant Contact with the “Long, Slow, SaaS Ramp of Death”

    http://businessofsoftware.org/2013/02/gail-goodman-constant-contact-how-to-negotiate-the-long-slow-saas-ramp-of-death/

    • http://inform.ly/ Dan Norris

      Thanks for commenting Wade. Yeah I’ve seen that talk it’s great. I often have to remind myself how many years some of these companies have been around. Contant Contact 15 years old, Aweber 15 years old, Infusionsoft 12 years old etc.

    • http://multimediamarketingshow.com/ Jake Hower

      Nice share Wade. I enjoyed that :)

  • http://www.alaininternetmarketing.com/ Alain Guillot

    Thank you for the reality check. By the way, I love your podcast.

    • http://inform.ly/ Dan Norris

      Ha thanks mate. I must do another episode!

  • Tim_M

    Putting a payment page up is the best test. As Joel from WP Engine says, he had people handing him money for a product he had not created. That was the test.

    I tested my MVP with a sign up for interest. Got <100 people.

    Built a MVP coffee subscription. Got 10 signups right away and away I went.

    • http://inform.ly/ Dan Norris

      Hey Tim it’s just not as simple as that unfortunately. Joel actually started with an email optin to be notified of launch. He then developed it for 7 weeks and then launched it with a post that got traction on hacker news. Unfortunately this isn’t the common experience if you aren’t an established founder. Your signup for interest of less than 100 people may have been a good result. Who knows? There are so many factors.

      Coffee subscription is different as I mention in the post you can run services as an MVP very easily it’s not the same with software.

  • karen Thompson

    Thank you Dan…I love your honesty…cheers

    • http://inform.ly/ Dan Norris

      Thankyou Karen

  • Greg Budgen

    Early feedback is definitely the key. Great post Dan. A lot of insightful info.

    • http://inform.ly/ Dan Norris

      Thanks Greg.

  • http://carstories.neue.ca/ Stephen Jeske

    Dan, nice to see someone tell it like it is! I second Barry’s opinion about survivor bias. The error of this kind of thinking has been well documented in stock trading literature. Unfortunately, in the startup field, no one wants to spoil the dream.

    • http://inform.ly/ Dan Norris

      Ha thanks Stephen. I’m happy to spoil it he he.

  • http://rymera.com.au/ Josh Kohlbach

    Hey Dan, great post mate and I hope this is a kick in the ass for the delusional among us that think you can build a lasting business in a couple of weeks. That might be the case for the lucky few, but for the rest of us it means days and nights and weekends iterating on the same idea and just working it like a mad man.

    I personally think your decision to scale it back to the first idea and stop chasing the viral startup myth is a good one. Stick with what works I say, you’re getting trials and paying customers and people aren’t leaving. All great signs.

    Myself and many others are following along, good luck with it all.

    • http://inform.ly/ Dan Norris

      Thanks man I’ve found myself even doing the same thing with the new service just tinkering with stuff when I don’t have to. The signs are looking good for it as well so I just need to be patient I think.

  • Dorai Thodla

    Thanks Dan. A wonderful post. I can definitely resonate with it.

    Dorai

    • http://inform.ly/ Dan Norris

      Thanks Dorai thanks for tweeting it too.

  • http://www.HireYourVirtualAssistant.com Owen McGab Enaohwo

    Dan Norris as I have told you in private I still feel that the Content Marketing ROI version of Inform.ly is the better product and the best problem to solve. Everyone and their mother online is talking about content marketing but there is no real tool out there that literally shows you the return on income on ones content marketing efforts. This tool would have been best for content marketing agencies who produce content for their customers so that they can show them the ROI, they would have been the best customers and secondly the next best customers would have been companies who are already heavily producing content.

    Secondly, when it comes to MVP, paying customers is not everything. Delivering real value to paying customers is what matters the most. Check out the Max Cameron of Kera interview on Mixergy.

    • http://inform.ly/ Dan Norris

      Hey Owen thanks mate re point 1. You may be right but in any case I don’t have the resources to pursue it. What you are describing is not easy. We built the page analytics stuff which was hard enough but there are 2 things that make it even harder. 1 is the fact that at some stage you have to tap into the financial system. Whether that be by adding dollar values to goals like Analytics does (which is only going to work for certain businesses and won’t be an accurate measure of lifetime value) or 2 tapping into the accounting systems at the back end (could be better but you’ll run into issues with associating visits with who the customer is, i.e. not really being able to track their origin. I think if you were to niche it down to one system like create a content analytics app for infusionsoft users there could be something there. Maybe I’ll do that one day but it’s no small job and I just ran out of time and resources. I thought it was better to focus on the one that I actually had paying customers for and did solve a real problem. Everyone gets excited about Analytics but in the end most small business owners don’t really want to pay to look at numbers, they have bigger problems. Most Analytics startups seem to focus on bigger companies which isn’t something I want to do. Look at how much KissMetrics is charging now.

      I don’t really understand point 2. You say paying customers isn’t everything but delivering value to paying customers is so I’m lost on that one. I’ll check out the interview.

      • http://digitaldomination.com/ Steve Fitzpatrick

        One thing I know about almost every mid-sized business is; They don’t measure.
        You and I and probably most people commenting have been schooled in measuring and tracking ROI. But from what I can see, nearly every company that employs us for Adwords campaigns or SEO or anything else don’t actually care about the numbers.

        Just having a tool that allows them to see some other metric would probably be enough for them to pay for and be happy. EG: This blog post had 15,000 reads! WOOHOO.

        • http://inform.ly/ Dan Norris

          Well Google Analytics does this, I guess what I was working on was trying to differentiate from that. I think there’s something for sure. Maybe someone else will do it. That product that Moz brought out certainly does look sexy. I’m not sure how effective it as at doing what we are talking about as I haven’t used it.

      • http://www.HireYourVirtualAssistant.com Owen McGab Enaohwo

        Yeah check out the interview to understand point #2. I am sure you will get it when you listen to it.

        Regarding point #1 you will definitely need to niche up. To get access to the financial system, for instance you can go after web apps who charge a recurring revenue who can simply link the revenue data to your tool or marketing agencies who create content for web apps.

        OR

        create the tool for folks who heavily create content and are users of tools like Hubspot, Marketo, Eloqua, InfusionSoft and so on. These tools already to an extent provide insight on the financial data.

        If you can build such a tool you literally could charge $99+ or more per month and folks will be willing to pay because the tool will relate revenue to specific content marketing effort.

  • http://digitaldomination.com/ Steve Fitzpatrick

    Great post Dan. It’s hard to admit when a venture doesn’t work out quite like you expected. I’ve had my own experiences but the good news is we aren’t alone.

    I recently covered off my thoughts on this in this post http://typicalbusiness.com/falling-in-love-with-your-ideas/ Overall, I love the way you pivoted – that’s often the biggest hurdle to get around. Falling in love with the original idea and not being willing to shift is probably one of the biggest mistakes I see regularly.

    • http://inform.ly/ Dan Norris

      Thanks Steve great post, particularly the fans stuff. It’s great to have people’s support but you can read too much into it I think. I’d never built anything that had ever been talked about by anyone of note before Informly so to have all of that external coverage and people talking was very exciting. But realistically not worth as much as a handful of keen customers.

      • http://digitaldomination.com/ Steve Fitzpatrick

        On that front Dan, you’ve achieved a lot more than most could imagine. I was saying to Jake Hower the other night that I think it’s just a matter of time before you hit the big one. Keep up the great work.

        • http://inform.ly/ Dan Norris

          Let’s hope so Steve

      • Stephen Kellett

        “But realistically not worth as much as a handful of keen customers.”

        Hit the nail on the head. My name is not well known and neither is the company. But look at the customer list (testimonials page) and we’re doing fine. You don’t need fame and ego stroking.

        You need customers. The noisy ones that complain and point out bugs and ask for features – they are your friends (even if at times they annoy you) – they are the ones that will push you to improve the quality and to take the feature set in the direction that provides the most benefit. No easy answer to which suggestions to accept – use you best instincts, trust your gut reaction to ideas if you evaluate each idea as yourself being the customer.

  • http://www.sentia.com.au/ Michael Cindric

    Great post Dan. I can agree with this as we launched our first product (doccyapp.com) 2 months ago and its hard work and progress is slow. I think everyone thinks its going to be a overnight success story for every startup but its a hard slog

    • http://inform.ly/ Dan Norris

      Hey Michael yeah I think so. Thanks for the comment and good luck with Doccy.

      • http://www.sentia.com.au/ Michael Cindric

        Thanks mate its a slog but worth it. Think its important for people to share their experiences that way ppl know what 99% of the services out there go through

  • John Romaine

    Solid post dude. I myself have invested into numerous projects and most of them have failed. Much like leveraging external properties to drive traffic to your websites, I think its smart to do likewise with projects (including the release of new products and services) Work all of them with the least amount of effort, find the ones that work, and ditch the others.

    • http://inform.ly/ Dan Norris

      Hey John thanks mate. I’m not sure if this is the answer to be honest. I think focus is better. Despite having 2 projects at the moment I think a better approach is choosing 1 and trying to figure it out. I might have failed a few times at it but I still think it’s the best way to go. It’s hard enough to do 1 thing well let alone 3 or 4 you end up spreading yourself too thin and don’t do any of them justice.

      • http://www.bringthedonuts.com/ John Romaine

        Fair point. Do you think it’s plausible to do 3-4 MVP’s, though? Like you did with WPLN.

        • http://inform.ly/ Dan Norris

          Sure, change the 1 idea as much as you need to is better than having 3-4 ideas I reckon.

          • http://www.bringthedonuts.com/ John Romaine

            This makes sense and is pretty much what Im doing with BTD. Everyone kept asking me what it was when I first got started. I kept replying “I dont know yet” – lol. Over time the long term plan is becoming much clearer. What’s great is that I can let the site evolve, I can pivot, shimmy and do whatever I need to in order to make it work. All at the one site – not several sites like Ive done in the past which has led to me being overwhelmed.

          • http://inform.ly/ Dan Norris

            I don’t know if this is the best strategy either. I think people have to be clear about what you are offering. If people don’t know what you do then it’s not a good sign. I’m not a fan of the name I think that’s a barrier regardless of what you turn it into.

            I think thinking about what you are doing as a ‘site’ is a problem too. You are either trying to build a business and solve problems or you aren’t. If your site supports it then great but the site isn’t the business. You have to remember that a lot of the guys in our circles (Schramko etc) all have businesses already and all they need is more people into the funnel. This is a fundamentally different position to what you are in.

  • Max Völkel

    Thanks for sharing this. I think you also made the “measuring the wrong things” mistake, which probably everybody makes to some degree. I tried to sum this up in a brief post:
    http://blog.xam.de/2013/07/validation-for-software-products-is-hard.html

    • http://inform.ly/ Dan Norris

      Thanks Max yeah I definitely made that mistake. I really like this post http://insideintercom.io/four-pricing-principals-to-never-forget/ particularly the chart on free users vs paid users and the sorts of feedback they give. I definitely fell into this trap.

      • Matthew Newton

        For what it’s worth, how good is that Inside Intercom blog. Nailing it always.

        • http://inform.ly/ Dan Norris

          Solid

  • Stephen Kellett

    Check out Bob Dorf on customer validation, business model canvas and lessions from history. He’s been there, blown it, made it, written about it, got the t-shirt too.

    http://businessofsoftware.org/2013/07/bob-dorf-co-author-startup-owners-manual-customer-development-the-science-of-acceleration-for-growth-businesses/

    FWIW, 6 weeks is not enough time. Not nearly enough.

    Most products are not overnight successes.

    When I started my current business I spent two years creating it (spare time – evenings + weekends and then testing it at my consulting day job), then 9 months full time, then launched 6 months after 9/11 (bad timing), then had the Iraq war (more bad timing – as each war starts businesses stop spending for 6 months or more). It’s a highly technical product with a geek audience. 3 months before first sale, probably 2 or 3 years before it was a self-sustaining business.

    Business is now in it’s 12th year. I have a nice business to run and I choose how and when I work.

    • http://www.HireYourVirtualAssistant.com Owen McGab Enaohwo

      Stephen Kellett

      I 100% agree with you!

      Software apps whereby one charges a recurring fee is a very very scalable business model! That said it is still a business and it takes time even years to fully work out all the kinks.

      Here are some steps to consider at the early stages of a software app startup and each one takes time.

      1) Figure out a specifically identifiable group of people to solve a problem for.

      2) Figure out the most burning pain/problem that they are willing to pay you to solve.

      3) Build the MVP solution that solves the problem

      4) Build a Repeatable and Predictable Customer Acquisition Process that can turn site visitors into leads, leads into trial users, and trial users into paying customers.

      5) Once the predictable marketing system of step 4 above is in place then it’s time to grow, pump as much paid traffic as you can to it

      As you can see these 5 steps highlighted so far only covering the first two stages of running a successful software business as suggested by David Cummings, co-founder of Pardot which recent got sold for about $100million to ExactTarget (<<<— see http://davidcummings.org/2013/07/05/the-four-stages-of-a-b2b-startup/)

      In total it was a 5.5 year process (<<<— see http://davidcummings.org/2012/11/17/key-moments-in-the-life-of-pardot/)

      • Greg Bardwell

        Skip the trial!!! If there is value they will pay. If not, they will not not. Their cost in not the price if there is value — it is it the changing the way they do business that is the hardest road block.

        Skip the free trial.

      • http://inform.ly/ Dan Norris

        Hi Owen this advice sounds great. I have found it’s not quite as simple as that for me once you start to make it happen. If you are able to pull it off then please send the guest post my way I’d love to have it on the blog!

        • http://www.HireYourVirtualAssistant.com Owen McGab Enaohwo

          It not meant to be simple, each stage comes with it’s own challenges and I’d gladly blog about it when the time comes. BTW in my case for SweetProcess it will be due to the combined efforts of my wonderful co-founders; Brian, Jervis and I

    • http://inform.ly/ Dan Norris

      Thanks Stephen this looks great I’ll check it out.

      Yeah the mistake with the content analytics product wasn’t killing it after 6 weeks it was starting it in the first place knowing full well I couldn’t dedicate enough time and resources to make it work. The other product I had worked on for 1 year full time and probably 2 more years part time, I had been going at it for a while.

      Great to hear about your persistence. I fear I don’t have the patience to pull that off and that definitely hurts me at times.

  • Kalen Jordan

    Awesome post, thanks Dan.

    That’s brutal that inform.ly didn’t work out.

    I still kind of can’t believe that it didn’t work out, I was following the progress through your email blasts and such, and it sounded like it was definitely going to work.

    Especially with all of that validation and positive feedback that you got.

    I don’t really know anything about building a successful product, but just wanted to share a few thoughts.

    I ran into something similar with the last product that I tried to build. Got a lot of great feedback, got a few sales, but then didn’t end up getting as many as I would have liked.

    One of the problems with the particular product that I was building was that the value proposition wasn’t a very easy sell. It was basically for upgrading software, and it was the kind of thing people hated to do in the Magento world and that they knew they had to do.

    But at the end of the day, unless there was a very specific bug that was losing them money every day, or a new feature they needed that would make them money, they really couldn’t be bothered to upgrade.

    And I can’t blame them, it makes sense.

    So one of the big things that I changed in the way I’m approaching building a product is that I’m hyper-focused on building something that creates value (and by value I mean money specifically), regardless of whether there’s necessarily a demand for it or whether the potential customers think they are going to want it.

    So the product I’m building now is actually able to calculate the ROI that it is driving. One of the nice things about working in Magento specifically (eCommerce platform) is that I’m able to do that pretty easily since sales are flowing through the platform, I know that’s not an option for every software product out there.

    And I’m not sure how much demand there is going to be for it, but because I know for a fact that this piece of software will make people money if they use it, I’m going to keep pushing even if there isn’t too much interest at first. Hopefully get up that slow ramp of death :)

    Thanks again for the post!

    • http://inform.ly/ Dan Norris

      Hey Kalen thanks mate yeah if you can find a situation where you are making more money than it costs people to use your service then it’s a very good start. It’s not a guarantee though because some people don’t act as logically as you or I do. But definitely it helps in selling something that you can prove that it will return it’s value.

  • Greg Bardwell

    One thing I learned a while back, is that just because there is a need, it does not mean there is a market. There is a lot of “cool, I could really use that.” There is much, much fewer, “cool, here is my credit card number.”

    I also learned start-ups and freelancers are NOT a viable market. Both are trying to trade their time for money and have no money to trade on saving time. Only want the free version and usually will not pay.

    By and large, you need to target a business that will pay to solve a pain point or need help on resources (time, but this needs to be compelling, as time ROI is a hard sell). Also, can target a hardcore consumer segment, such as a high priced hobby.

    I thought your content marketing product may have had legs… but it was too technical to make work for most of us. And your price point was too low. Remember the price is NOT the sticker price … that is almost incidental if there is value. The real price is changing the way we do things.

    Good luck.

    • http://inform.ly/ Dan Norris

      Thanks Greg good insights.

    • Stephen Kellett

      I can verify these points. Along our journey we’ve had various startups (a few big names too, now defunct) that said they’d purchase. Very few ever do and even when they do, they don’t purchase much.

      You want customers that are not startups but real businesses looking to be more efficient, more effective and that value what you provide.

  • Akshay Bhatt

    Sounds great Dan. As a someone who has gone through stages you have already passed, your insights seems intuitive to me. There are times in our product cycle that a mvp is not just enough, specially if you are trying to sell something. While giving response people tend to behave differently then people who are real buyers of product. That being said, the thing I like about MVP is speed, a must. That helps not going into micro-engineering every aspect and produce first improve later approach. May be in 2015, someone will comeup with a new strategy, and then in 2017 and so on. What’s the point ? We got to do something right at the moment and evolve naturally.

    • http://inform.ly/ Dan Norris

      Thanks mate agreed.

  • John Peterson

    Excellent post Dan. What is funny is that I randomly found Inform.ly because I had an itch to scratch. I think your trials and tribulations reflect what most startups go through. It’s called reality, and doesn’t make the news. In addition to experience, you also bring some great business skills to the table. Thanks for sharing that with everyone.

    • http://inform.ly/ Dan Norris

      Thanks John I appreciate that.

  • AbilityMatrix

    Great post. I do not think that anything went wrong. The only thing that is way too overhyped is how success stories do their business and get traction. I do not think that they are more conscious on how they got there than Google or Facebook. (if they knew it they could repeat their original success which they could not, in fact.) And see how many times they fail. Especially Google: they are awesome at failing again and again and again.

    There is no formula for success. We are experimenting with defining the key emotional impacts users are looking for. Focusing on functions solely we push our customers to focus on functions that create the perceived quality customers are looking for (this is not the same as UX). We are pretty successful with it but still it helps only to define the potential that can be achieved with the product. It also helps to stay focused on what you need to deliver.

    Still I have not seen anything that will tell you when to enter the market and what is your exact market. You can forget lean canvas and all other overhyped bullshit methods. The secret sauce is still out there for someone to define.

  • Che Cooper

    Great post Dan and gives me some stuff to think about before I start my next adventure.

    • http://inform.ly/ Dan Norris

      Awesome mate great to hear. Let me know how it goes for you.

  • JulioBarros

    Hi Dan,

    Great post with lots of thought provoking points.

    I’ve often wondered if what you end up testing is a particular landing page or messaging or ads vs actual product potential? There are so many variables in play that you could have a great product but the test results may not be well correlated.

    Hope that makes some sense.

    Thanks again for writing it.

    Julio

    • http://inform.ly/ Dan Norris

      Hey Julio thanks mate yeah I did a bunch of split tests for new features as well I didn’t include those above. I didn’t really get a whole lot of validation from those either, I had tests for new features under-perform conversions on the existing pages, changing from features to benefits having minimal impact etc. I guess in the end you have to make your own decisions based on what is generally incomplete data.

  • trcull

    Dan,

    I’d thought of something just like the original Inform.ly just a month or so after you did and actually shied away because I didn’t think I could do a better job than you were. I’ve been following you all along. It’s kind of sobering to see behind that curtain–thanks for taking the bullet for us!

    Thanks also for pointing out the survivor bias in everything we read. I used to buy it all hook, line, and sinker and it made me feel terrible. But thanks to honest, transparent posts like yours and others, I realized I’m actually better off than some of the people I’ve been admiring all along and there isn’t any magic formula to overnight success.

    Anyway, I’m about to pull the trigger on another product instead. Keeping my fingers crossed:
    http://www.thedwick.com/2013/07/a-tale-of-two-products/

    I’d love to buy you a beer or several. I think we’d have some good stories to swap.

    • http://inform.ly/ Dan Norris

      Thanks man sounds cool, let me know when you launch something I’ll have a play!

  • mworleyjr

    Hey Dan thanks for your transparency in this post. I have had some of the same frustrations with the validation process with a software product that I have been working on with a client. I think one of the main problems is the definition of “Validation” as the process is a little bit more complicated than sending out a tweet and having a response. One guy that you have to read is Peter Eerling’s blog, especially his series on going from Services to Product. This post alone (http://blog.eerlings.com/index.php/2013/04/04/how-solving-a-real-pain-makes-us-already-1000-a-month-even-before-the-big-launch/) answers much of your software frustrations from Inform.ly.

    Look forward to listening to your podcast interview on Start Up’s for the Rest of Us.

    • http://inform.ly/ Dan Norris

      Thanks Mike I’m checking this one out now.

  • BobPA

    I think the idea with MVP is the features that are implemented are highly polished and well designed. You’re just leaving out features that aren’t really needed to solve the problem (eye candy, bells-and-whistles etc)

    • http://inform.ly/ Dan Norris

      Easier said than done in my experience.

  • Pingback: Sun Up, Sun Down, It’s the September Best of the Web! | Firepole Marketing Blog

  • Pingback: Worried someone's going to steal your idea? Here are 4 things you should worry about instead

  • TerrenceYang

    Awesome post, thanks. And Steve Blank may say you have a good lifestyle business, not quite a startup.

    • http://wpcurve.com/ Dan Norris

      Thanks mate. Glad you liked it.

      I remember when Jason Calacanis accused DHH of having a lifestyle business. Pretty solid lifestyle business that one turned out to be.

  • Osman Sheikh

    Always appreciate your honesty when it comes to startups Dan. Good read.

    • http://wpcurve.com/ Dan Norris

      Thanks Osman.

  • http://www.entrepreneursforachange.com/ Lorna Li

    “Targeted Surveys don’t work” – I shit you not. I slaved away for weeks to create a product based on my audience survey answers, released a beta version for $17 and 9 people bought it.

    • http://wpcurve.com/ Dan Norris

      9 sounds good. I had about 20 people tell me they’d pay for something and I think 1 of them did. Surveys for the lose.

  • Troy Dean

    Hey Dan – I agree that email optins and bet a testers don’t mean you have an idea worth pursuing. I built WP Elevation into a six figure business with steady recurring monthly revenue by throwing up a landing page on Unbounce.com and selling tickets to a webinar ($97 p/head) before I had built the webinar. This webinar went on to become the business accelerator program that it is today based on feedback from those original webinar attendees.

    The lesson here is that people paying you money is a far better indicator that you’re onto something than an email address.

    I presented how I did this at the Phoenix Lean Startup MeetUp via Google Hangout yesterday:

    http://www.youtube.com/watch?v=G8aHcXI56kc

    Keep up the awesomeness mate.

    • http://wpcurve.com/ Dan Norris

      Hey Troy this is cool, I guess the point I’d make (which I also made in the article) is that validation is pretty clear when the results are obvious. Like if you have an audience and you sell them a webinar and you get way more signups than you expected then happy days. The problem is a lot of people don’t get that clearly positive result. In that case it’s not quite as simple. Having an audience is a huge asset too as we’ve discovered with all of the things we’ve launched this year.

  • http://www.grangewebdesign.com Kieran Daly

    Belated comment – listened to you on Foolish Adventure – WPCurve looks great – looks like there is plenty of niche there for this sort of micro support. by the way your twitter links dont work in footer :-)

    • http://wpcurve.com/ Dan Norris

      Thanks Kieran I’ll check the footer.