Brad's Books and Organizations

Books

Books

Organizations

Organizations

Hi, I’m Brad Feld, a managing director at the Foundry Group who lives in Boulder, Colorado. I invest in software and Internet companies around the US, run marathons and read a lot.

« swipe left for tags/categories

swipe right to go back »

I’m Done With Private Beta

Comments (44)

I was going to call this post "Private Beta is Bullshit" but then I realized I might be wrong.  Rather than decide, I’m looking for reasons to change my mind.  Please help me.  In the spirit of thinking out loud on my blog, I’m going to go through a history lesson from my perspective to frame the problem.

When I started writing commercial software in the 1980′s, there was a well-defined "beta process."  Your first beta was actually called an alpha – when you had your friends and a few lead users play around with your stuff which was guaranteed to break every few minutes.  But they were a good source of much needed early feedback and testing.  Then came beta – you shipped your disks out to a wider audience,  including a bunch of people you didn’t know but who were interested in your product, and they banged away looking for bugs.  You had a bug collecting and management process (if you were really cutting edge you even had a BBS for this) and while there wasn’t a code freeze, you spent all of your time fixing bugs and hardening / optimizing the code.  If you had a complex system, you started shipping release candidates (RCs); less complex went straight to a release (GA).  Inevitably some bugs were found and a bug fix version (v x.01) was released within a week or two.  At this point you started working on the next version (v x+1.0); if there were meaningful bugs still in v x you did small incremental releases (v x.02) on the previous code base.

This approach worked well when you shipped bits on disks.  The rise of the commercial Internet didn’t change this approach much other than ultimately eliminate the need to ship disks as your users could now download the bits directly. 

The rise of the web and web-based applications in the later 1990′s (1997 on) did change this as it was now trivial to "push" a new version of your app to the web site.  Some companies, especially popular consumer sites and large commercial sites, did very limited testing internally and relied on their users to help shake down the web app.  Others had a beta.website.com version (or equivalent) where limited (and often brave) users played around with the app before it went in production.  In all cases, the length of time of the dev/test/production loop varied widely.

At some point, Google popularized the idea of an extended beta.  This was a release version that had the beta label on it which is supposed to indicate to people that it’s an early version that is still evolving.  Amazingly, some apps like Gmail (or Docs or Calendar), seem to never lose their beta label while others like Reader and Photos have dropped them already.  At some point, "beta" stopped really meaning anything other than "we’ve launched and we probably have a lot of bugs still so beware of using us for mission critical stuff."

With the rise of the Web 2.0 apps, beta became the new black and every app launched with a beta label, regardless of its maturity (e.g. a whole bunch of them were alphas.)  Here’s where the problem emerged.  At some point every beta got reviewed by a series of web sites led by TechCrunch (well – not every one – but the ones that managed to rise above the ever increasing noise.)  When they got written up, many of them inevitably ran into The First 25,000 Users Are Irrelevant problem (which builds on Josh Kopelman’s seminal post titled 53,651which might be updated to be called 791K.)  During this experience, many sites simply crash based on the sudden load as they weren’t built to handle the scale or peak load.

Boom – a new invention occurred.  This one is called "private beta" and now means "we are early and buggy and can’t handle your scale, but we want you to try us anyway when we are ready for you."  I’ve grown to hate this as it’s really an alpha.  For whatever reason, companies are either afraid to call an alpha an alpha or they don’t know what an alpha is.  For a web app, operational scale is an important part of the shift from alpha to beta, although as we’ve found with apps like Twitter, users can be incredibly forgiving with scale problems (noisy – but forgiving).

So – why not get rid of the "private beta" label and call all of these things alphas.  Alphas can be private – or even public – but they create another emotional and conceptual barrier between "stuff that’s built but not ready for prime time" (alpha), "stuff that getting close but still needs to be pounded on by real users and might go down" (beta), and "stuff that’s released" (v x.0).  That seems a lot more clear to me than "private beta", "beta" (which might last forever), and ultimately v x.0. 

In the grand scheme of things this might simply end up in "Brad Pet Peeve" land, but recently it’s been bothering me more than my other pet peeves so it feels like there’s something here.  Call me out if there isn’t or pile on if there is.

  • Joe Wheeler

    apologies for the double post….got a connection error message

  • Joe Wheeler

    apologies for the double post….got a connection error message

  • JungleDave

    I'm a regular reader here.. first time poster :)
    This topic was one I felt compelled to reply to as I think you're on track, but I do think there is still a place for private betas – which I'll describe below.
    You're right that what many companies call a 'private beta' really is an alpha – incomplete with a long list of known issues (including scalability). A true beta should be feature complete with a minimum of known issues and represent the final phase of testing before release. So where does a private beta fit in? In my experience, when you transition from internal/F&F testing to public testing there are always a number of issues that come up quickly that were not found internally – things like conflicts with 3rd party software or plugins, firewalls, browser versions, dependencies on internal servers, etc.
    If you go straight to a public beta, you run the risk of these issues being seen by thousands of users before you can fix them. A short private beta period with a few hundred users across a wide mix of environments helps shake out these issues.
    As an example – we just launched a private beta for the Workgroup Edition of Jungle Disk. Within a few hours we had identified an issue that was affecting a fair number of the users that we had missed in testing because it was specific to their Amazon S3 accounts. Had we launched straight to public beta it's likely that several thousand users would have run into this problem and had a bad first experience with the product. Our private beta period will likely be short – two to three weeks max, but that additional testing will give us confidence that when we open up the public beta we haven't missed any embarrassing issues.

    • http://www.feld.com Brad Feld

      Dave – while this makes sense, it still feels like it’s alpha land to me. Presumably you are selectively letting in some people you don’t know but have expressed interest. However, you are still throttling (or metering) the invites to control the usage to test for your obvious edge cases.

      I guess there an “alpha”, “private beta”, “public beta”, “release” pacing. My struggle is that I’m confused as to how long private betas last and some of them now stretch on for months!

      • http://blog.jungledisk.com JungleDave

        Well, here's one differentiator in my book – if no issues came up in the private beta I would ship the same code as the public beta. You would never do that from alpha->beta. Of course many folks have abused the “beta” and “private beta” labels, so that when they are used in their traditional sense folks such as yourself aren't sure what to expect.

  • http://davidduey.typepad.com David Duey

    From a user's perspective I've always assumed that Private Beta means the product is beyond alpha (i.e. not terribly buggy) but the ability to scale is still in question. I don't mind the Private Beta designation because it implies that the developers are being thoughtful about product roll-out.

    • http://www.coloradostartups.com David G. Cohen

      you don't need to make it private to communicate that though.

  • http://joblogs.cc Steve Ireland

    Marketers of small scale apps are also using “beta” as a cooler version of “new”; which makes beta less useful than it was. Companies could help themselves by broadening their lexicon or just communicate better to manage expectations.

    • http://www.feld.com Brad Feld

      Totally agree!

  • http://blogs.newsgator.com/inbox Nick Harris

    I agree with Jungle Dave and also with Nick Bradbury’s comments about beta software in the world of desktop applications now a days (http://nick.typepad.com/blog/2008/05/beta-buggy.h

    With Inbox 3.0, I did a short private beta period which I used to both shake out configuration bugs as Jungle Dave points out (especially important with a Outlook add-in) as well as flushing out exactly what the new feature set would look like. This proved to be very helpful since I ended up adding a handful of “helper” features to compliment the new features in 3.0.

    With what Nick B. points out, I had a private tester who seemed more than a little ticked off that Inbox 3.0 wouldn’t load in his Outlook configuration and that it took me a few days to identify the issue and fix it. If I had released publicly, I would have left a bad taste with thousands of users instead of just a few.

    Which brings up bug reporting. Managing bug reports in a beta period is hard enough, and having the same issue reported by thousands of users makes it even tougher. Being able to flush out the easy to find / common use case bugs with a small audience gives you the chance to clean those up before the avalanche of bug reports that come with a public beta.

  • Kimm Viebrock

    I was going to say that you're wrong straight out, until I understood that you were suggesting that these private betas be called alphas. I'll give you that, but from a support professional's experience, my take on it is that even if it's been used inappropriately recently, private beta is still an important stage in development.

    If scalability is a question, then certainly that is one of the things you want to test, so private beta shouldn't be used to mask a lack of scalability. There are plenty of other reasons, though, for private beta and part of it has to do with managing expectations and risk, which can't happen as well if it's a public beta, even with a more forgiving public. I've always counseled that it's important to define a set number of desired beta participants and a set list of criteria for what makes a good beta participant and that no beta testing happens without falling in line with those parameters. A public beta can't do that and whether your customers are paying with money, eyeballs, or goodwill, it's not smart to squander that.

    In private beta, you're in shakedown mode and not just for the code. You're testing other post-release aspects of your operation too and how they might be impacted by the release. You're growing your support knowledge and expertise in real-world scenarios, not the la-la land of ideal lab conditions that occur during alpha. You don't really want support to be learning that stuff (and showing their lack of knowledge) in the full fray. As JungleDave pointed out, you're also looking for more issues to address by getting a better cross-section of users than you might have had during the alpha phase. In fact, in recruiting private beta participants, I look for operational platform diversity. Sure it's more trivial these days to roll out a fix for a problem you might discover during this phase, but do you really want to have do expose that problem in front of the entire user population, especially if it's one of those ugly gotchas?

    I also like to recruit beta participants based on how they use the product (are they using it in depth, breadth or some other way unique that gives me more data than I would have otherwise?) and their willingness and availability to partner not only on feedback but on troubleshooting too, without risking damage to the customer/provider relationship. This also can't be said of a public beta. Sure, you'll probably get a diverse set of feedback, but that doesn't guarantee you'll get people who experience issues AND are willing to work with you appropriately to sort them out. Plus, you'll get all sorts of people who get noisy over stuff you already know about or don't want to know about and you'll have to deal with that as well as the people who just go snark on you behind your backs. This is not managing risk well.

    To treat it any other way than a shakedown cruise is to be pushing (alpha) code out publicly before it's time, expecting beta code to solve issues when you don't actually know if that's true or not or, as Steve points out, using the term 'beta' inappropriately as the new New for marketing.

    I realize that some aspects of this may be less true for a smaller operation, but that doesn't make it untrue, even for SAAS. I've had many occasions proving the most important reason to come in at 6am is to be available for the onslaught of customer yelps when code got pushed out overnight without letting us know in support. Private betas done right greatly minimize that and it keeps customers and other parts of your operation much happier when it works the way it's supposed to.

    • http://www.feld.com Brad Feld

      Kimm – outstanding thoughts. This is really helpful perspective – there isn't anything here I can argue with.

  • http://www.awhere.com Jim Pollock

    One of the issues is the fluidity of the semantics. I started writing commercial software in 1981 and in several companies I've always pushed and adhered to a methodology and nomenclature of:

    Engineering test: internal – engineering team tests their own code
    Alpha: internal – test audience expands to people that didn't write the code (other engineering teams, marketing, customer support staff)
    Beta: external – limited number of customers, early adopters

    We rigidly defined alpha as an expanded INTERNAL audience. Beta was for the user community.

    Of course in the 80's, my software experience was scientific/engineering desktop software so there were no scalability issues. (Except could we handle the phone calls if we shipped buggy software.) Which is why in my most successful experiences, we let customer support have full veto power on any rev shipment decisions. They were the most impacted group in the company if bad software went out.

    So I am going to vociferously agree with JungleDave. In the “new world of software” scalability and interaction with other software plug-ins/layers is very important and nearly impossible to test internally. You NEED a friendly external audience to excercise it. You want hundreds of testers, maybe thousands, but not hundreds of thousands which could destroy your reputation or ability to sell the REAL product and inundate your development team with noise.

    I'm all for the private beta… when it's used correctly.

    Jim

    • Kimm Viebrock

      Thanks for saying it much more succinctly than I did.

      Oh, and based on the other comments I see here that also make sense, maybe what needs to go away is the public beta?

    • http://www.feld.com Brad Feld

      The difference here is the notion that “alpha” is the internal test. I've always used alpha for an external test with a small number of friendly and active users. The internal stuff shouldn't get to be called alpha since the organization is still “self testing” (important, but not in the release cycle).

      The more I read the comments, the more I realize this is semantics (as you correctly point out in your first sentence.) My struggle is that there isn't uniform agreement on what each of the alpha / beta / whatever means, so different companies use them different ways. While the notion of an IEEE standard for a naming convention is nonsense, the variance in the different naming usages is making me tired.

  • DH

    I'm in agreement with Steve. I think a lot of these companies release their products as Beta as it sounds more cool than Alpha. Personally, when I see a new startup releasing a product with an alpha label, I don't even bother trying it out unless there's a compelling reason to do so, as it's probably bug ridden and won't work properly.

    There's still a place for private betas (or more accurately, alphas) though. As Dave said, there's always issues when going from internal testing to public testing, and a private beta / alpha version can help to weed some of these out before releasing to the general population.

  • http://mikepk.com mikepk

    We had this debate inside Grazr, what does “Beta” mean in the new Web 2.0 world? The truth is, it doesn't mean much. I'm an ex systems / hardware guy and my partner has lots of experience with shipping software during the bits-on-disk days. We put grazr out there as a public alpha initially, then transitioned to beta thinking in the old paradigm of software development. We then removed the beta badge once we thought of it as “release”. We even used version numbers when we had major changes, at each major point release we've had a “beta”. We think of our current version as a beta of version number 3, but most people don't know what that means. Google set the bar for beta to mean “totally functional except we change things occasionally”. Most people looking at web services only have a very general concept of the engineering alpha, beta, release development phases so I suspect that for many, beta just means “it works like gmail”.

    I had considered writing a blog post on this a long time ago but it never left draft form. The other interesting thing I see is the transition of users mental model of software from something physical. This is directly analogous to the change that occurred for documents on the web. Originally the only mental model was physical paper, so when people had to grapple with the constantly changing, dynamic nature of the web, they resorted to putting “under construction” on every page. Eventually people realized that “under construction” was meaningless, the page is never 'done', and that the medium itself was fluid. I think the same thing is happening with web applications. “Beta” is one way for people to deal with the transition from a static to a dynamic medium for the delivery of functionality.

    Another quick point is that I think you're also overlooking what I believe is one of the reasons private betas are in vogue right now, marketing. Creating an artificial sense of scarcity with special passes and only allowing a select few to gain access can generate buzz for a product. Requiring people to ask for “membership to the beta club” also engages them early. Being a data nerd, I'd love to know if this early interaction actually increases the user engagement to the product. Posts like “Anyone have invites for XXX” can in and of themselves pique interest. I think this fad will pass though as people experience private beta fatigue.

  • Eddie LeBreton

    Honestly, Brad, as a long-time reader I enjoyed the post, but when I step back it just looks like a semantic argument to me. As Web 2.0 continues to evolve, so will the lexicon. True, there is ambiguity in the phrasing at the moment, but I imagine it will sort itself out and, even if it doesn't, I doubt anyone is being misled by the disparate use of “alpha,” “beta,” “limited beta,” etc.

  • http://www.burnertrouble.com Martin Edic

    Private beta looks like nothing more than an extended QA session, something some services may have to do given limited resources. We're more along the 37 signals approach of 'get it out there and improve constantly', based on input.
    This is my second SaaS experience (I'm a marketing guy) and the first acted like old-school, taking months or years to do updates. This one does them almost constantly with one tenth of the resources and we're not talking just bug fixes. New features and improved UI based on feedback. It's great after the last experience.

    So here's a different take on the beta question: We offer a free version and those users act like our beta users (there's lots more of them and they're early adopters). Our paid customers tend to suggest the more sophisticated improvements based on their real working requirements. Both are very valuable for different reasons.

  • http://blog.jeffreymcmanus.com/ Jeffrey McManus

    I came to the conclusion that private betas were bullshit a while back. But, after having done a few, I'm now at the point where I won't do a release without a private beta (or alpha, if you prefer, it's really just semantics). The people who have access to the alpha will be limited to those who I know will provide constructive feedback. For my next project, that's going to be close friends and prospective investors.

  • http://www.emaildashboard.com Deva Hazarika

    A lot of great comments already. I wrote a very similar post in 2006 and linked to posts with the same issue from 2004 – this is nothing new and unlikely to go away. I do think that it's two different issues. One is things not really beta being called beta. The other is how to reach users for a real beta in a controlled fashion. More detailed discussion and one potential solution on my blog post on this topic: http://www.emaildashboard.com/2008/06/the-beta-ex

  • http://www.protrails.com Dave Schwartz

    Jim Pollock brought up semantics which was my visceral reaction. No reason to get upset about how someone describes their QA / UI optimization process. To each his own and if the product turns out stable and desirable, then who cares what the beta was called! And as a beta partner for Google Friend Connect right now, I can tell you that regardless of what Google calls it, what we're involved with is a QA and Dev request process which is turning out great. Hope all is well!

  • John Ball

    Clearly, you hit a nerve.

    I've been involved in the delivery of software products since the (very) early 80's. Alpha was always internal test. Beta was really the opportunity to test “nearly final code” and the preparedness of the organization to deliver and support the code.

    Scalability is not a new issue. Neither is integration with other applications, APIs, widgets, etc.

    What is “newish” is the growing desire to push more responsibility out to the end user for nearly every facet of product evaluation, education, and use. There are striking similarities to simple cost reduction efforts underway in nearly every business. The underlying truth here is that many web companies; too often funded on the belief that it only takes a few hundred k to launch a product, severely underfund the efforts required for going to market in responsible fashion and apparently feel compelled to share their shortcomings with the prospective market.

    Like you Brad, I've decided to abandon participation in private betas. I soon discovered there was no need to hide from the paparazzi after enrolling in a private beta and my ego was ill served.

  • Shan Sinha

    Brad- I agree that there is a certain “fad” element associated with using the term beta. And no one seems to want to use the term “alpha”.

    However, let me propose one legitimate use of the term “private beta” that I think has emerged as particularly unique to our day and time. It is how we are expecting to use our private beta when we launch in early fall.

    Because we are no longer “shipping bits”, we have an opportunity to get software up and running more quickly and test value propositions earlier. In the “shipping bits” world, it was very difficult & costly to get a high scale test of your core value proposition. Surveys? Focus groups? All inadequate.

    However, in today's day and time, it is exceedingly easy to get up and running one value proposition and test whether that is the correct one, at some levels of scale. By running a private beta, you have an opportunity to pull together a group of users who can really help you decide what direction your product needs to go.

    That's not to imply that the level of quality shouldn't be at or near “beta quality”. One example to point to is Xobni. They started off as email analytics and moved to Fast Email Search as positioning. It's harder to do that when you are in the “public” phase.

    Testing that value proposition and iterating is to me the key distinction on the use of a private beta.

    • http://www.feld.com Brad Feld

      Shan – that’s a good example. If I synthesize several of the comments defending “private beta”, they all come back to the notion that there is a role for something between “alpha” and “public beta”. I still struggle with calling it “private beta”, but as someone who used Xobni during this time period, it was a logical positioning.

  • Don Diegol

    Fear of failure – I think I have read about this recently?

  • http://joewheeler.tumblr.com Joe Wheeler

    Brad I think like most of Web 2.0 there is a fluffy ring to “private beta” vs. alpha. I don't get it, and like you, would rather call it what it is.

    What do you think fueled the Web 2.0 rush to launch with a beta label by every startup out there? people seeking funding, being the first to launch in a niche, ease of launching a product, lack of understanding, front page of Digg, Arrington love…?

  • http://joewheeler.tumblr.com Joe Wheeler

    Brad I think like most of Web 2.0 there is a fluffy ring to "private beta" vs. alpha. I don't get it, and like you, would rather call it what it is.

    What do you think fueled the Web 2.0 rush to launch with a beta label by every startup out there? people seeking funding, being the first to launch in a niche, ease of launching a product, lack of understanding, front page of Digg, Arrington love…?

  • http://midtownninja.com MidtownNinja

    I think the allure of 'private beta' lies in the private, not in the beta. Anything that creates an air of quasi exclusivity is bound to get written about and talked about (echoing mikepk above). Plus, it's a reliable source of leads that a startup can contact once it has its stuff together a bit more.

  • Shane

    You have to blame Google for this. They've almost single handedly screwed up the term beta. Gmail 'beta' that still asks me to 'invite' anyone I email to join Gmail a year after it's been publicly available to everyone?!? It's a total pet peeve of mine too.

    I also think that web 2.0 startups should focus more on finding the right early alpha users and working with them more closely. Directly to Josh's point, the early adopter tech crowd aren't mainstream consumers. It's easy to get these folks to sign up and test your product and give you feedback that… “it should be integrated with Twitter and manage my blog posts, and RSS feeds”…when most of the folks in your target audience have never used any of these.

    Maybe someone should create a community ranking service that allows people to quickly vote on the 'status' of an online app. Alpha – Beta – Release – Or Prime Time. Then we could ridicule Google when they call their Prime Time apps Beta, and put most of the Web 2.0 startups back in their place when they call an Alpha product a Beta.

    I'll put some code together this weekend, and try to get a Private Beta out by Monday ;)

  • todd sawicki

    Brad –
    In my experience with web apps – alpha's are really early stage concepts meant to illustrate a concept or feature in a live web setting. Private beta's are meant as the first working release while public beta's are meant as often short term, fix the bugs and test scaling in public (goog is the one violating this rule not the other way around). And the reality is that public betas are dying in my view so what's really bullshit are public betas as the private beta is to me true to the way the term was used in the good old software days (limited release, constant revisions, etc.)

    • http://www.feld.com Brad Feld

      That’s an interesting perspective, especially given the Googleness of “public beta”. Several folks have commented that it’s “public beta” that has gotten perverted by the Google usage – that’s an interesting thing to ponder.

  • http://www.direct2costarica.net owen

    it's new for me!

  • http://sikanrong.com sikanrong

    awesome article

  • http://sikanrong.com sikanrong

    awesome article

  • http://treadaway.typepad.com Chris Treadaway

    I think this is a pet peeve at most Brad…

    Alpha means that the app could pretty much be rewritten based on feedback, and it likely doesn't have all necessary features. Closed Beta = testing real life usage in a sandbox. Beta = release.

    We've learned a ton in our Closed Beta release of MinutesNotice that, quite frankly, I'm very glad we didn't learn in a Beta. Perception of what we're doing would be awful, we'd probably have been panned already in major blogs, and we'd be sunk.

    Someone above nailed it when they said to blame Google. They are to blame, not folks trying to get their apps off the ground.

  • http://furrier.org John Furrier

    Brad, been a while since i posted a comment on your blog.. this is one sweet post… your'e right on

    i'm 42 so i'm with you. there isn't anything wrong with coming out of the closet with 'private beta' which to me is pre alpha. It's an internet platform – hell why not…

    I do think private beta confuses the capital markets…

    years ago (not to long ago) – you pitched a vc with powerpoint- aka slideware
    now – you pitch with private beta

    private beta is the the new 'slideware' –

    relevance, real customers, scale matter – not many vcs get that…

    great post!

  • Shane Jones

    One clear line that can be drawn between Alpha and Beta in my mind is wether or not your data will persist through to the final release. Previous alpha releases that I've worked on have had a policy that all data and accounts will be wiped clean before entering the beta / general release process.

    Not a big deal for traditional 'installed' software, but it's an important factor for web services. Has anyone come across a legitimate site that asked people to play around, provide feedback, but don't expect your data to persist past this release…I've never seen it.

  • http://kosso.wordpress.com Kosso

    Hi Brad. Phreadz had a 'clandestine alfalfa mode' and we're now in the 'closed beetroot era'

    Yes, I'm poking fun at all these 'alpha/beta' sites out there. But I definitely have the need to do this in this way – mainly due to securing the necessary funds to build out the hardware and bandwidth architecture required to reduce the chance of it falling flat on its face if I swung the doors wide open – for more testing.

    There are sites out there which are fully open to the whole world, requiring no Wonka ticket invites to join, which still carry the 'Alpha Release' slogan. I'll agree that this irritates me.

    My product works. And when I can afford to, I'll be opening it up, for all to use. And there will be no vegetation or vapor in sight.

  • http://www.feld.com Brad Feld

    from an email comment: “Simple: the CEO/CTO of an early stage startup is far more concerned with Impressing the Board than he is with the quality of the product. I've written a lot of code for startups — you've funded some of them — and these guys are scared to death of telling you and the board that there are too many bugs to release it right now. If you asked about bug find rates and fix rates, bugs found in alpha testing, beta testing, etc., then maybe these guys would do these things. In short, it's your own fault.”

    • Steve Bergstein

      My experience with small organizations is that they rarely have good metrics. They're too busy doing things like writing code to analyze data like bug find/fix rates.

      OTOH, perhaps board members could drive better behavior by insisting on getting the right facts.

  • http://www.coloradostartups.com David G. Cohen

    The real question is whether or not doing a beta “privately” makes any sense. I think it can in certain situations. But for me the peeve is more about startups who use it as a marketing gimmick. getting an “invitation” is supposed to seem really hard. they're trying to make you feel special. the reality is that if almost all (98% of?) private betas were simply open betas, their user counts would be only marginally different. and without all the hoops they make their users jump through. there's no “cool factor” to this unless you really do have a million users on your waiting list. the upside is for startups that do have a long wait list, it can be really useful to ensure you can scale over time, doing so slowly as you let users “in”. So, for me the answer is just let users in and stick the beta label on it along with appropriate warnings. then if you really have too much demand before you're ready (and thus are in the 2%), you can start taking names and letting them in slowly. otherwise, you're simply creating an annoyance for your potential users which you might also call the dreaded “friction”. but with popular up and coming services like brightkite, it does seem to work well for them. people are seeking the rare invitation (i've sent about 100 out). so like most things in the world, the answer is of course “it depends.”

  • Edward H.

    I agree with the posters that think this is as much about marketing as it is about defining a new or real distinction. Private means exclusive. Alpha may be construed a laborious. As you correctly point out, the word beta has lost some of its meaning, or it has evolved to mean something else. Something new was needed.

    I love this post though. I studied English instead of something useful. I cringe when I hear people describe something as “ironic,” as 90% of the time they are describing something that is “coincidental.” While there is nothing nefarious about Private Beta I am sure, changing the meaning of words for the purposes of marketing is dangerous. Ask George Orwell :-)

    Amen for being anal.

    • http://www.feld.com Brad Feld

      Yeah – I always confuse ironic and coincidental along with then / than and accept / except.

  • http://newmanva.com/blog arinewman

    Some great insights here. As a company currently in private beta this is topic near and dear to our hearts. At Filtrbox we did a private beta for a number of reasons and it has worked out well for us. A lot of this is semantics, I agree, but there is truth in the core statement.

    When Filtrbox began the transition from friends and family testing to opening it up to a broader audience, we were in no way ready for the mass public and our goal wasn't to see how many users we could handle, it was to learn what needed to be improved before going public. We leaned A LOT, without getting slammed or dismissed as a “not ready for primetime” app by the public when we knew we weren't ready. We are very focused on delivering a usable, quality product that solves problems and going right into public mode wasn't not going to align well with that objective. We also did successfully build up a bit of demand, and our early users have been engaged and helpful in testing the product and giving feedback.

    When we go public, there will be no “beta” label on the product, and we will stand behind Filtrbox as a commercial application.

    Perhaps we should have called it “Alpha” or “preview” to more accurately reflect the app's current state at the time but in the end I feel like we actually evolved from alpha to beta while operating the service under the “private beta” banner and it worked out.

    Related: My partner Tom put up a blog post last year about Google's widget funding program…seems even finance is on the beta bandwagon. http://tomchikoore.wordpress.com/2007/06/29/you-a

  • http://www.stylehop.com/blog/ David Reinke

    At StyleHop we are staying “private alpha” – it actually took a bit of discussion for us to land there since it's not the current convention. Glad to see others are thinking the same way we are.

  • http://intensedebate.com/people/jungledave48931 jungledave48931

    I'm a regular reader here.. first time poster :)
    This topic was one I felt compelled to reply to as I think you're on track, but I do think there is still a place for private betas – which I'll describe below.
    You're right that what many companies call a 'private beta' really is an alpha – incomplete with a long list of known issues (including scalability). A true beta should be feature complete with a minimum of known issues and represent the final phase of testing before release. So where does a private beta fit in? In my experience, when you transition from internal/F&F testing to public testing there are always a number of issues that come up quickly that were not found internally – things like conflicts with 3rd party software or plugins, firewalls, browser versions, dependencies on internal servers, etc.
    If you go straight to a public beta, you run the risk of these issues being seen by thousands of users before you can fix them. A short private beta period with a few hundred users across a wide mix of environments helps shake out these issues.
    As an example – we just launched a private beta for the Workgroup Edition of Jungle Disk. Within a few hours we had identified an issue that was affecting a fair number of the users that we had missed in testing because it was specific to their Amazon S3 accounts. Had we launched straight to public beta it's likely that several thousand users would have run into this problem and had a bad first experience with the product. Our private beta period will likely be short – two to three weeks max, but that additional testing will give us confidence that when we open up the public beta we haven't missed any embarrassing issues.

  • Jim Pollock

    One of the issues is the fluidity of the semantics. I started writing commercial software in 1981 and in several companies I've always pushed and adhered to a methodology and nomenclature of:

    Engineering test: internal – engineering team tests their own code
    Alpha: internal – test audience expands to people that didn't write the code (other engineering teams, marketing, customer support staff)
    Beta: external – limited number of customers, early adopters

    We rigidly defined alpha as an expanded INTERNAL audience. Beta was for the user community.

    Of course in the 80's, my software experience was scientific/engineering desktop software so there were no scalability issues. (Except could we handle the phone calls if we shipped buggy software.) Which is why in my most successful experiences, we let customer support have full veto power on any rev shipment decisions. They were the most impacted group in the company if bad software went out.

    So I am going to vociferously agree with JungleDave. In the "new world of software" scalability and interaction with other software plug-ins/layers is very important and nearly impossible to test internally. You NEED a friendly external audience to excercise it. You want hundreds of testers, maybe thousands, but not hundreds of thousands which could destroy your reputation or ability to sell the REAL product and inundate your development team with noise.

    I'm all for the private beta… when it's used correctly.

    Jim

  • http://intensedebate.com/people/david_duey david_duey

    From a user's perspective I've always assumed that Private Beta means the product is beyond alpha (i.e. not terribly buggy) but the ability to scale is still in question. I don't mind the Private Beta designation because it implies that the developers are being thoughtful about product roll-out.

  • http://intensedebate.com/people/eddie_lebr12311 eddie_lebr12311

    Honestly, Brad, as a long-time reader I enjoyed the post, but when I step back it just looks like a semantic argument to me. As Web 2.0 continues to evolve, so will the lexicon. True, there is ambiguity in the phrasing at the moment, but I imagine it will sort itself out and, even if it doesn't, I doubt anyone is being misled by the disparate use of "alpha," "beta," "limited beta," etc.

  • Dave Schwartz

    Jim Pollock brought up semantics which was my visceral reaction. No reason to get upset about how someone describes their QA / UI optimization process. To each his own and if the product turns out stable and desirable, then who cares what the beta was called! And as a beta partner for Google Friend Connect right now, I can tell you that regardless of what Google calls it, what we're involved with is a QA and Dev request process which is turning out great. Hope all is well!

  • http://intensedebate.com/people/kimm_viebro1980 kimm_viebro1980

    I was going to say that you're wrong straight out, until I understood that you were suggesting that these private betas be called alphas. I'll give you that, but from a support professional's experience, my take on it is that even if it's been used inappropriately recently, private beta is still an important stage in development.

    If scalability is a question, then certainly that is one of the things you want to test, so private beta shouldn't be used to mask a lack of scalability. There are plenty of other reasons, though, for private beta and part of it has to do with managing expectations and risk, which can't happen as well if it's a public beta, even with a more forgiving public. I've always counseled that it's important to define a set number of desired beta participants and a set list of criteria for what makes a good beta participant and that no beta testing happens without falling in line with those parameters. A public beta can't do that and whether your customers are paying with money, eyeballs, or goodwill, it's not smart to squander that.

    In private beta, you're in shakedown mode and not just for the code. You're testing other post-release aspects of your operation too and how they might be impacted by the release. You're growing your support knowledge and expertise in real-world scenarios, not the la-la land of ideal lab conditions that occur during alpha. You don't really want support to be learning that stuff (and showing their lack of knowledge) in the full fray. As JungleDave pointed out, you're also looking for more issues to address by getting a better cross-section of users than you might have had during the alpha phase. In fact, in recruiting private beta participants, I look for operational platform diversity. Sure it's more trivial these days to roll out a fix for a problem you might discover during this phase, but do you really want to have do expose that problem in front of the entire user population, especially if it's one of those ugly gotchas?

    I also like to recruit beta participants based on how they use the product (are they using it in depth, breadth or some other way unique that gives me more data than I would have otherwise?) and their willingness and availability to partner not only on feedback but on troubleshooting too, without risking damage to the customer/provider relationship. This also can't be said of a public beta. Sure, you'll probably get a diverse set of feedback, but that doesn't guarantee you'll get people who experience issues AND are willing to work with you appropriately to sort them out. Plus, you'll get all sorts of people who get noisy over stuff you already know about or don't want to know about and you'll have to deal with that as well as the people who just go snark on you behind your backs. This is not managing risk well.

    To treat it any other way than a shakedown cruise is to be pushing (alpha) code out publicly before it's time, expecting beta code to solve issues when you don't actually know if that's true or not or, as Steve points out, using the term 'beta' inappropriately as the new New for marketing.

    I realize that some aspects of this may be less true for a smaller operation, but that doesn't make it untrue, even for SAAS. I've had many occasions proving the most important reason to come in at 6am is to be available for the onslaught of customer yelps when code got pushed out overnight without letting us know in support. Private betas done right greatly minimize that and it keeps customers and other parts of your operation much happier when it works the way it's supposed to.

  • Nick Harris

    I agree with Jungle Dave and also with Nick Bradbury’s comments about beta software in the world of desktop applications now a days (http://nick.typepad.com/blog/2008/05/beta-buggy.h

    With Inbox 3.0, I did a short private beta period which I used to both shake out configuration bugs as Jungle Dave points out (especially important with a Outlook add-in) as well as flushing out exactly what the new feature set would look like. This proved to be very helpful since I ended up adding a handful of “helper” features to compliment the new features in 3.0.

    With what Nick B. points out, I had a private tester who seemed more than a little ticked off that Inbox 3.0 wouldn’t load in his Outlook configuration and that it took me a few days to identify the issue and fix it. If I had released publicly, I would have left a bad taste with thousands of users instead of just a few.

    Which brings up bug reporting. Managing bug reports in a beta period is hard enough, and having the same issue reported by thousands of users makes it even tougher. Being able to flush out the easy to find / common use case bugs with a small audience gives you the chance to clean those up before the avalanche of bug reports that come with a public beta.

  • Steve Ireland

    Marketers of small scale apps are also using "beta" as a cooler version of "new"; which makes beta less useful than it was. Companies could help themselves by broadening their lexicon or just communicate better to manage expectations.

  • DH

    I'm in agreement with Steve. I think a lot of these companies release their products as Beta as it sounds more cool than Alpha. Personally, when I see a new startup releasing a product with an alpha label, I don't even bother trying it out unless there's a compelling reason to do so, as it's probably bug ridden and won't work properly.

    There's still a place for private betas (or more accurately, alphas) though. As Dave said, there's always issues when going from internal testing to public testing, and a private beta / alpha version can help to weed some of these out before releasing to the general population.

  • http://intensedebate.com/people/bfeld bfeld

    Totally agree!

  • http://intensedebate.com/people/martin_edi18731 martin_edi18731

    Private beta looks like nothing more than an extended QA session, something some services may have to do given limited resources. We're more along the 37 signals approach of 'get it out there and improve constantly', based on input.
    This is my second SaaS experience (I'm a marketing guy) and the first acted like old-school, taking months or years to do updates. This one does them almost constantly with one tenth of the resources and we're not talking just bug fixes. New features and improved UI based on feedback. It's great after the last experience.

    So here's a different take on the beta question: We offer a free version and those users act like our beta users (there's lots more of them and they're early adopters). Our paid customers tend to suggest the more sophisticated improvements based on their real working requirements. Both are very valuable for different reasons.

  • John Ball

    Clearly, you hit a nerve.

    I've been involved in the delivery of software products since the (very) early 80's. Alpha was always internal test. Beta was really the opportunity to test "nearly final code" and the preparedness of the organization to deliver and support the code.

    Scalability is not a new issue. Neither is integration with other applications, APIs, widgets, etc.

    What is "newish" is the growing desire to push more responsibility out to the end user for nearly every facet of product evaluation, education, and use. There are striking similarities to simple cost reduction efforts underway in nearly every business. The underlying truth here is that many web companies; too often funded on the belief that it only takes a few hundred k to launch a product, severely underfund the efforts required for going to market in responsible fashion and apparently feel compelled to share their shortcomings with the prospective market.

    Like you Brad, I've decided to abandon participation in private betas. I soon discovered there was no need to hide from the paparazzi after enrolling in a private beta and my ego was ill served.

  • Jeffrey McManus

    I came to the conclusion that private betas were bullshit a while back. But, after having done a few, I'm now at the point where I won't do a release without a private beta (or alpha, if you prefer, it's really just semantics). The people who have access to the alpha will be limited to those who I know will provide constructive feedback. For my next project, that's going to be close friends and prospective investors.

  • http://intensedebate.com/people/shan_sinha41091 shan_sinha41091

    Brad- I agree that there is a certain "fad" element associated with using the term beta. And no one seems to want to use the term "alpha".

    However, let me propose one legitimate use of the term "private beta" that I think has emerged as particularly unique to our day and time. It is how we are expecting to use our private beta when we launch in early fall.

    Because we are no longer "shipping bits", we have an opportunity to get software up and running more quickly and test value propositions earlier. In the "shipping bits" world, it was very difficult & costly to get a high scale test of your core value proposition. Surveys? Focus groups? All inadequate.

    However, in today's day and time, it is exceedingly easy to get up and running one value proposition and test whether that is the correct one, at some levels of scale. By running a private beta, you have an opportunity to pull together a group of users who can really help you decide what direction your product needs to go.

    That's not to imply that the level of quality shouldn't be at or near "beta quality". One example to point to is Xobni. They started off as email analytics and moved to Fast Email Search as positioning. It's harder to do that when you are in the "public" phase.

    Testing that value proposition and iterating is to me the key distinction on the use of a private beta.

  • MidtownNinja

    I think the allure of 'private beta' lies in the private, not in the beta. Anything that creates an air of quasi exclusivity is bound to get written about and talked about (echoing mikepk above). Plus, it's a reliable source of leads that a startup can contact once it has its stuff together a bit more.

  • Shane

    You have to blame Google for this. They've almost single handedly screwed up the term beta. Gmail 'beta' that still asks me to 'invite' anyone I email to join Gmail a year after it's been publicly available to everyone?!? It's a total pet peeve of mine too.

    I also think that web 2.0 startups should focus more on finding the right early alpha users and working with them more closely. Directly to Josh's point, the early adopter tech crowd aren't mainstream consumers. It's easy to get these folks to sign up and test your product and give you feedback that… "it should be integrated with Twitter and manage my blog posts, and RSS feeds"…when most of the folks in your target audience have never used any of these.

    Maybe someone should create a community ranking service that allows people to quickly vote on the 'status' of an online app. Alpha – Beta – Release – Or Prime Time. Then we could ridicule Google when they call their Prime Time apps Beta, and put most of the Web 2.0 startups back in their place when they call an Alpha product a Beta.

    I'll put some code together this weekend, and try to get a Private Beta out by Monday ;)

  • Don Diegol

    Fear of failure – I think I have read about this recently?

  • http://intensedebate.com/people/todd_sawic49431 todd_sawic49431

    Brad –
    In my experience with web apps – alpha's are really early stage concepts meant to illustrate a concept or feature in a live web setting. Private beta's are meant as the first working release while public beta's are meant as often short term, fix the bugs and test scaling in public (goog is the one violating this rule not the other way around). And the reality is that public betas are dying in my view so what's really bullshit are public betas as the private beta is to me true to the way the term was used in the good old software days (limited release, constant revisions, etc.)

  • Kosso

    Hi Brad. Phreadz had a 'clandestine alfalfa mode' and we're now in the 'closed beetroot era'

    Yes, I'm poking fun at all these 'alpha/beta' sites out there. But I definitely have the need to do this in this way – mainly due to securing the necessary funds to build out the hardware and bandwidth architecture required to reduce the chance of it falling flat on its face if I swung the doors wide open – for more testing.

    There are sites out there which are fully open to the whole world, requiring no Wonka ticket invites to join, which still carry the 'Alpha Release' slogan. I'll agree that this irritates me.

    My product works. And when I can afford to, I'll be opening it up, for all to use. And there will be no vegetation or vapor in sight.

  • http://intensedebate.com/people/mikepk mikepk

    We had this debate inside Grazr, what does "Beta" mean in the new Web 2.0 world? The truth is, it doesn't mean much. I'm an ex systems / hardware guy and my partner has lots of experience with shipping software during the bits-on-disk days. We put grazr out there as a public alpha initially, then transitioned to beta thinking in the old paradigm of software development. We then removed the beta badge once we thought of it as "release". We even used version numbers when we had major changes, at each major point release we've had a "beta". We think of our current version as a beta of version number 3, but most people don't know what that means. Google set the bar for beta to mean "totally functional except we change things occasionally". Most people looking at web services only have a very general concept of the engineering alpha, beta, release development phases so I suspect that for many, beta just means "it works like gmail".

    I had considered writing a blog post on this a long time ago but it never left draft form. The other interesting thing I see is the transition of users mental model of software from something physical. This is directly analogous to the change that occurred for documents on the web. Originally the only mental model was physical paper, so when people had to grapple with the constantly changing, dynamic nature of the web, they resorted to putting "under construction" on every page. Eventually people realized that "under construction" was meaningless, the page is never 'done', and that the medium itself was fluid. I think the same thing is happening with web applications. "Beta" is one way for people to deal with the transition from a static to a dynamic medium for the delivery of functionality.

    Another quick point is that I think you're also overlooking what I believe is one of the reasons private betas are in vogue right now, marketing. Creating an artificial sense of scarcity with special passes and only allowing a select few to gain access can generate buzz for a product. Requiring people to ask for "membership to the beta club" also engages them early. Being a data nerd, I'd love to know if this early interaction actually increases the user engagement to the product. Posts like "Anyone have invites for XXX" can in and of themselves pique interest. I think this fad will pass though as people experience private beta fatigue.

  • http://intensedebate.com/people/kimm_viebro1980 kimm_viebro1980

    Thanks for saying it much more succinctly than I did.

    Oh, and based on the other comments I see here that also make sense, maybe what needs to go away is the public beta?

  • http://intensedebate.com/people/bfeld bfeld

    from an email comment: "Simple: the CEO/CTO of an early stage startup is far more concerned with Impressing the Board than he is with the quality of the product. I've written a lot of code for startups — you've funded some of them — and these guys are scared to death of telling you and the board that there are too many bugs to release it right now. If you asked about bug find rates and fix rates, bugs found in alpha testing, beta testing, etc., then maybe these guys would do these things. In short, it's your own fault."

  • http://intensedebate.com/people/shane_jone42421 shane_jone42421

    One clear line that can be drawn between Alpha and Beta in my mind is wether or not your data will persist through to the final release. Previous alpha releases that I've worked on have had a policy that all data and accounts will be wiped clean before entering the beta / general release process.

    Not a big deal for traditional 'installed' software, but it's an important factor for web services. Has anyone come across a legitimate site that asked people to play around, provide feedback, but don't expect your data to persist past this release…I've never seen it.

  • Chris Treadaway

    I think this is a pet peeve at most Brad…

    Alpha means that the app could pretty much be rewritten based on feedback, and it likely doesn't have all necessary features. Closed Beta = testing real life usage in a sandbox. Beta = release.

    We've learned a ton in our Closed Beta release of MinutesNotice that, quite frankly, I'm very glad we didn't learn in a Beta. Perception of what we're doing would be awful, we'd probably have been panned already in major blogs, and we'd be sunk.

    Someone above nailed it when they said to blame Google. They are to blame, not folks trying to get their apps off the ground.

  • http://intensedebate.com/people/deva_hazari2084 deva_hazari2084

    A lot of great comments already. I wrote a very similar post in 2006 and linked to posts with the same issue from 2004 – this is nothing new and unlikely to go away. I do think that it's two different issues. One is things not really beta being called beta. The other is how to reach users for a real beta in a controlled fashion. More detailed discussion and one potential solution on my blog post on this topic: http://www.emaildashboard.com/2008/06/the-beta-ex

  • http://intensedebate.com/people/steve_bergs2127 steve_bergs2127

    My experience with small organizations is that they rarely have good metrics. They're too busy doing things like writing code to analyze data like bug find/fix rates.

    OTOH, perhaps board members could drive better behavior by insisting on getting the right facts.

  • owen

    it's new for me!

  • John Furrier

    Brad, been a while since i posted a comment on your blog.. this is one sweet post… your'e right on

    i'm 42 so i'm with you. there isn't anything wrong with coming out of the closet with 'private beta' which to me is pre alpha. It's an internet platform – hell why not…

    I do think private beta confuses the capital markets…

    years ago (not to long ago) – you pitched a vc with powerpoint- aka slideware
    now – you pitch with private beta

    private beta is the the new 'slideware' –

    relevance, real customers, scale matter – not many vcs get that…

    great post!

  • http://intensedebate.com/people/dgcohen dgcohen

    The real question is whether or not doing a beta "privately" makes any sense. I think it can in certain situations. But for me the peeve is more about startups who use it as a marketing gimmick. getting an "invitation" is supposed to seem really hard. they're trying to make you feel special. the reality is that if almost all (98% of?) private betas were simply open betas, their user counts would be only marginally different. and without all the hoops they make their users jump through. there's no "cool factor" to this unless you really do have a million users on your waiting list. the upside is for startups that do have a long wait list, it can be really useful to ensure you can scale over time, doing so slowly as you let users "in". So, for me the answer is just let users in and stick the beta label on it along with appropriate warnings. then if you really have too much demand before you're ready (and thus are in the 2%), you can start taking names and letting them in slowly. otherwise, you're simply creating an annoyance for your potential users which you might also call the dreaded "friction". but with popular up and coming services like brightkite, it does seem to work well for them. people are seeking the rare invitation (i've sent about 100 out). so like most things in the world, the answer is of course "it depends."

  • http://intensedebate.com/people/bfeld bfeld

    The difference here is the notion that "alpha" is the internal test. I've always used alpha for an external test with a small number of friendly and active users. The internal stuff shouldn't get to be called alpha since the organization is still "self testing" (important, but not in the release cycle).

    The more I read the comments, the more I realize this is semantics (as you correctly point out in your first sentence.) My struggle is that there isn't uniform agreement on what each of the alpha / beta / whatever means, so different companies use them different ways. While the notion of an IEEE standard for a naming convention is nonsense, the variance in the different naming usages is making me tired.

  • http://intensedebate.com/people/arinewman120 arinewman120

    Some great insights here. As a company currently in private beta this is topic near and dear to our hearts. At Filtrbox we did a private beta for a number of reasons and it has worked out well for us. A lot of this is semantics, I agree, but there is truth in the core statement.

    When Filtrbox began the transition from friends and family testing to opening it up to a broader audience, we were in no way ready for the mass public and our goal wasn't to see how many users we could handle, it was to learn what needed to be improved before going public. We leaned A LOT, without getting slammed or dismissed as a "not ready for primetime" app by the public when we knew we weren't ready. We are very focused on delivering a usable, quality product that solves problems and going right into public mode wasn't not going to align well with that objective. We also did successfully build up a bit of demand, and our early users have been engaged and helpful in testing the product and giving feedback.

    When we go public, there will be no "beta" label on the product, and we will stand behind Filtrbox as a commercial application.

    Perhaps we should have called it "Alpha" or "preview" to more accurately reflect the app's current state at the time but in the end I feel like we actually evolved from alpha to beta while operating the service under the "private beta" banner and it worked out.

    Related: My partner Tom put up a blog post last year about Google's widget funding program…seems even finance is on the beta bandwagon. http://tomchikoore.wordpress.com/2007/06/29/you-a

  • http://intensedebate.com/people/dgcohen dgcohen

    you don't need to make it private to communicate that though.

  • http://intensedebate.com/people/bfeld bfeld

    Kimm – outstanding thoughts. This is really helpful perspective – there isn't anything here I can argue with.

  • http://intensedebate.com/people/bfeld bfeld

    Yeah – I always confuse ironic and coincidental along with then / than and accept / except.

  • http://intensedebate.com/people/edward_h50271 edward_h50271

    I agree with the posters that think this is as much about marketing as it is about defining a new or real distinction. Private means exclusive. Alpha may be construed a laborious. As you correctly point out, the word beta has lost some of its meaning, or it has evolved to mean something else. Something new was needed.

    I love this post though. I studied English instead of something useful. I cringe when I hear people describe something as "ironic," as 90% of the time they are describing something that is "coincidental." While there is nothing nefarious about Private Beta I am sure, changing the meaning of words for the purposes of marketing is dangerous. Ask George Orwell :-)

    Amen for being anal.

  • JungleDave

    Well, here's one differentiator in my book – if no issues came up in the private beta I would ship the same code as the public beta. You would never do that from alpha->beta. Of course many folks have abused the "beta" and "private beta" labels, so that when they are used in their traditional sense folks such as yourself aren't sure what to expect.

  • http://intensedebate.com/people/bfeld bfeld

    Dave – while this makes sense, it still feels like it’s alpha land to me. Presumably you are selectively letting in some people you don’t know but have expressed interest. However, you are still throttling (or metering) the invites to control the usage to test for your obvious edge cases.

    I guess there an “alpha”, “private beta”, “public beta”, “release” pacing. My struggle is that I’m confused as to how long private betas last and some of them now stretch on for months!

  • http://intensedebate.com/people/bfeld bfeld

    That’s an interesting perspective, especially given the Googleness of “public beta”. Several folks have commented that it’s “public beta” that has gotten perverted by the Google usage – that’s an interesting thing to ponder.

  • http://intensedebate.com/people/bfeld bfeld

    Shan – that’s a good example. If I synthesize several of the comments defending “private beta”, they all come back to the notion that there is a role for something between “alpha” and “public beta”. I still struggle with calling it “private beta”, but as someone who used Xobni during this time period, it was a logical positioning.

  • David Reinke

    At StyleHop we are staying "private alpha" – it actually took a bit of discussion for us to land there since it's not the current convention. Glad to see others are thinking the same way we are.

  • http://twitter.com/fithappypro @fithappypro

    I realize I'm a little late to join this discussion, but I just happened to be googling 'Private Beta' and this post ranked high on the return. As I respect a lot of your work Brad.

    I'm just wondering if 'Private Beta' could be a good way to get a little market research done as a small start-up?Even if it does work perfectly, could you not use a 'Private Beta' (or whatever you like to call it) to test the waters before a full launch? It seems to me that it would be which is why I'm leaning towards giving it a try.

  • http://twitter.com/fithappypro @fithappypro

    I realize I'm a little late to join this discussion, but I just happened to be googling 'Private Beta' and this post ranked high on the return. As I respect a lot of your work Brad.

    I'm just wondering if 'Private Beta' could be a good way to get a little market research done as a small start-up?Even if it does work perfectly, could you not use a 'Private Beta' (or whatever you like to call it) to test the waters before a full launch? It seems to me that it would be which is why I'm leaning towards giving it a try.

Build something great with me