The Lazy Rando Review Policy…

19 05 2011

Testing stuff the hard way...

I figured it was about time I published a review policy I could point people to should the topic be of interest to them. I buy about 90% of the products reviewed on this site. Companies give me the other 10% of the product to review [ie. the Donkey Boxx or Scruzol].

Why don’t I buy 100% of the stuff I review?

Well partly because I have a limited budget to spend on gear and partly because I may well already have something that fills the roll of the product in question. The best reviews are those that contrast and compare similar products. For example I own a Berthoud rando bar bag which cost me $300 [ouch!].  I’d love to try out 3 or 4 more rando bags from various companies [Velo Orange, Swift Industries, etc..], but paying $200-$300 a pop with a decaleur for something I already own is not likely to happen. However, if the companies with similar products want to send me something to test it makes some great reviews possible. You get a review that wouldn’t otherwise happen, they get some PR for their product and I get to retire before I’m 90! Sounds like a win-win-win to me…=-)

Can you trust a review based on free product?

First off I’d say don’t trust any review from anyone 100% without critically thinking about it. Evaluate their expertise in the field, their motivation to mislead you and their track record. The more a reviewer/site is sponsored and has advertisers to answer to the less the reader’s needs will be at the forefront of their thoughts. Does the site have anything negative to say about products or is everything they touch AWESOME!! ? Do they have a tendency to always slam products that don’t fit their narrow definition of good? Personally I find it pretty easy to spot the lame reviews that are on the net. Looking at the Donkey Boxx I got recently the value of the product is ~$30. I’ll end up spending around 20hrs of my life writing a couple of reviews, photographing the box and talking to folks about it afterwards. That doesn’t include all the time Sharon spends using it, but we’ll ignore that for now. So I get about $1.50/hr value out of the deal. If I was doing it for the money I’d be better off working at 7-11. More expensive products tend to be more complex and I invest more time in the review process so my dollar/hr value doesn’t sky-rocket for say a $120 bike light. So far nobody has paid me to do a review and I’m not interested in getting advertisers for this blog.

Does that make me objective?

No. I don’t pretend to be objective. I have my own biases. I know people in the bike & kiteboard industries by their first names. Heck I know some of their kid’s first names. On the other hand I’ve been blogging almost daily for 6yrs now and my content sees over a million views a year [thanks for reading BTW!…=-)] so I’ve got a track record for generating useful content and a reputation to protect. How valuable/applicable a particular review is to your situation is something you have to determine.

Stuff I buy:

  • if I buy it I’m going to review it and post the review without any contact with the company who made/sold it. That’s not because I’m a jerk, but I already spend a lot of my free time on this blog and engaging every company whose product I review would add significantly to my workload.
  • I don’t moderate my blog’s comments [other than for SPAM and nastiness] so a vendor is always welcome to leave a comment in response to my review.
  • in fact anyone who wants to say something about a review can leave a comment. Whether you agree or disagree I want to hear it and your comment will help other folks evaluate my review.
  • if I reviewed a product you sell and you find my review objectionable you can also email me off the blog and I’ll talk to you about your concerns. I’m totally open to amending a review with new material if the discussion leads me to believe I made a mistake or if there is more to the story.
  • Internet reviews can linger on Google for a long long time so I want to be fair to everyone and report things accurately.
  • if you don’t like a review of your product and don’t have a sensible rebuttal I’m sorry, but that’s too bad I won’t edit a review to make you happy. You should just be happy I bought your product!

Stuff I get for free:
  • If you want to send me something and I’m pretty sure I’ll hate it I’ll tell you upfront and suggest we skip the review. If you insist I’ll do the review, but don’t get mad at me if it turns out badly.
  • If I do a review of a free item and there is a significant problem I’ll talk to the vendor and incorporate their feedback before the review is published. To be clear that doesn’t mean I won’t talk about the negative issue, but it does mean they’ll get their chance to give their side of the story right in the review. I figure that’s fair.
  • I don’t return review items. The cost of doing the review in terms of my time is already significant. Pulling the product from a bike, packaging it and shipping it back to a vendor adds significantly to the cost of a review. Assuming my time is only worth $20/hr [I’m a professional engineer and professional project manager so hopefully you’ll agree I’m being conservative] and a review takes 20hrs – that’s $400 of my time spent on the review. I’m not eager to add more time to that total. Keep in mind this blog earns me $0/yr in advertising or other payments. A vendor gets a considerable amount of exposure for their product in exchange for the wholesale cost of the item. Again I think that’s fair.
Review Period

I try to review stuff for a minimum of 12 months so that I can get a comprehensive impression of it and report my short, medium and longer term observations. Where possible I’ll just keep using and reporting on products. It’s hard to find quality longterm reviews online and let’s be honest – it’s way easier to be stoked by something on Day 1 than it is on Day 1000. I’d like to think this is one way I can really add value for people that find my reviews.

A Note to Vendors

If you are thinking about sending me something to review and trying to decide if it’s worth it I’d suggest you evaluate the situation critically just like I think a reader of this blog should evaluate a review. My content sees about 1 million views/year targeted at bike tourists, bike commuters, randonneurs, kiteboarders/surfers. Many of my reviews come up on the first page of related Google searches. Consider the wholesale cost of your item + shipping cost vs. the potential benefits of having content featuring your product in front of this particular audience. I can’t promise you a positive review, but I can promise you a fair & comprehensive review. I can also offer longterm feedback on your product which can be useful for product upgrades/development. I’m always open to discussing a potential review so feel free to drop me a line and chat.



8 responses

19 05 2011

Great article. Like you I buy my own gear and evaluate it for the bike and the purpose it is used for. I tend to try things out for months, then if it still doesn’t please me , I move onto something else and sell it off as a bargain to someone else, who might just find it suits them. I have one question, can you suggest a good 2 person tent, 3 seasons, good wet weather characteristics, dual entry, freestanding, quick erecting time and reasonable weight ? great blog don’t stop writing it.

19 05 2011

@Paul – if you want an ultralight tent – try the MSR Hubbs HUbba – there is a nice optional vestibule for wet weather trips that you can leave at home if you want a lighter configuration.

The other option that comes to mind is the REI Half Dome 2 which is not as light, but isn’t bad and it’s lower cost:

There are lots of other good choices if you aren’t weight/brand obsessed, but these two are a good place to start.

19 05 2011

paul, if you are in the US, the REI Half Dome has been my go to tent for 3 years (we only get out backpacking in the back country for about 45 nights a year) but the abuse it has taken without complaint has been impressive. it’s got room for gear in the vestibules and withstood 17 hours of two adults trapped inside because of pouring rain without allowing condensation or leaks (just stir craziness). Anyway, it’s on sale on-line for $179 and in the stores at $134.99 (get the footprint for $18.99)–ours weighs in at just over 5 lbs with the upgraded stakes.

Best of luck,

19 05 2011
Ty Smith

@Paul – I have the half dome 2 tent myself. In fact, I have the first generation and it works just fine for my wife and myself. I am 6’1″ 194 lbs and she is 5’0″ 102 and we have plenty of room. Quick set-up, light weight, and mininal condensation issues. Also agree with Thomas re: the footprint. Definitley get that as well.

19 05 2011

Vik, not that you need more work, but it might be an interesting project to take a look at the products that you reviewed three to six years ago and post some brief comments on how your views may have changed (or not) since then, with links to the original reviews.

19 05 2011

@Foraker – I’ve thought of that and do go that route sometimes. For example my Ortlieb Velocity backpack updates:

I lack time to do that with all the stuff I’ve reviewed, but I try to work my way through the key items or if someone asks about a product in particular.

I’m also trying to keep the blog balanced so it’s not 100% reviews.

19 05 2011

this is one of the best blogs on the net, period.

12 06 2011

Vik, I agree with Alan. And I appreciate the reviews and random thoughts, and always take with a grain of salt (no offense) any review or advice I read online until I know for myself and think critically about the issue. I think the most insightful part of this post is that your time has value, $20/hr is very conservative, and I think the time reviewing something and writing it up can be a lot to give. I, for one, appreciate it.

Another perspective: I estimate my free time at a $35 value. So the cash you save me (and others) by avoiding pitfalls/mistakes and expanding my thinking …. well, that’s priceless.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: