I'm in the process of testing changes to the our monitoring engine, specifically the NotifyWire search engine. You see, when monitoring Craigslist for your searches, NotifyWire doesn't actually run each individual search on Craigslist. Why? Well two reasons. First, it would require a lot more hits against Craigslist's servers, and second, it would actually be slower (there's a long story there).

Instead, NotifyWire just get's all the posts and runs your searches using our own search engine. This search engine was designed to mimic the engine used by Craigslist, so the results should match. I thought it was all working OK, but have had some reports recently about missing alerts...which I take very seriously.

I've probably spent 40-50 hours this week working on a test harness and fixing issues. The test harness uses real searches from NotifyWire users and runs the search against Craigslist. I then run the same results through the NotifyWire search engine, expecting each one evaluate to a positive match. However, on the first try I calculated a 10% mismatch. I also created a test for false positives that generated a 2% mismatch. Obviously this was unacceptable, so I got to work.

The results were so off, that I'm now wondering if Craigslist had changed there search engine because it was matching results in a much more flexible manner and some logical conditions that were broken now seemed to be fixed. For instance, you couldn't group negative terms together in parentheses before, but that works now. Here's another interesting tidbit - Craigslist's displays ads as they were written, but behind the scenes indexes an automatically spell checked version of the ad. I found multiple instances where Craigslist returned results that didn't have a required term properly spelled, so I'm thinking they don't do spell checking on the fly for each search, but index the spell checked results will all HTML tags removed.

I tried to run an open source spell checker against the ads before searching them, but the results were mixed. Out of ten cases found, the spell checker only got the right word once. For the amount of processing required, it wasn't worth it, so I pulled that out. In the end, the NotifyWire search engine now has a >.1% false negative rate and a 0% false positive rate. The remaining false negatives (should have matched but didn't) are due to spelling and the fact that one user's job search was based on word "compensation" that appears in the very bottom of a job ad, outside of the user provided content. Craigslist only provides the user generated content in their RSS, so the text is never found, but Craigslist includes it in their own search index.

I'm really happy with the results of re-writing the NotifyWire search engine.  I expect to release the new version on Sunday. Also in this new version will be navigation hotkeys, something that's been missing for far too long and really makes the NotifyWire application even more fun to use.

CLHandbook.com is like the missing instruction set for Craigslist.  If you’re new to Craigslist and looking for practical Craigslist tips, I can’t think of a better place to start.  Their guides can walk you through posting your first ad and the precautions you should take when buying and selling on Craigslist.

If you’re looking for the latest Craigslist news, they’ve also got a great blog.  Check it out.

I'm told Zhu Zhu pets are the hot holiday gift this year.  I guess I'm getting old because I really don't understand why and I want to say something like "kids these days..." 

A quick search of Craigslist yielded all sorts of options at a wide range of prices.  If you can't find these toys at the store because they're out of stock, then give Craigslist a try.

Watch out for scammers though.  Never buy anything unless you meet the seller face to face and can inspect the item in person.

I haven't been blogging much lately because I'm hard at work on the new version of NotifyWire.  While working on a couple issues, I noticed a really interesting change in Craigslist's html. 

It looks like Craigslist is going to start giving image previews as part of their browsing experience.  This is totally speculation, but look at what they've included in the HTML in front of each item description:

<span class="ih" id="images:3mc3oe3pa5Pb5S65R79an8ac3b5bbcc6e1752.jpg">

Looking at the HTML above, you can see the "id" of the empty span is specifying the address of the first image for the ad.  Why are they doing it this way?  I'm guessing they plan to do image previews using jQuery, but we'll see.

BTW, the image for the ad above can be found at http://images.craigslist.org/3mc3oe3pa5Pb5S65R79an8ac3b5bbcc6e1752.jpg, so you can see how they'd use the information in the "id" attribute and jQuery to dynamically make the image URL. 

Posting an ad on Craigslist inevitably leads to spam responses.  Some responses are scams looking to dupe you out of your money or your property (or both).  A twist on that theme sending out ads, this type of spam only steals your time.

Soon you can expect to see strange offers on Amazon.com in response to your Craigslist ads thanks to this project on GetAFreelancer.com.  If you see a response to your Craigslist ad that contains a link to Amazon.com send the contents to Amazon using this form.  Amazon won't put up with this kind of behavior from affiliates and will close that affiliate account in a crack addict’s heartbeat. 

Before Yahoo! froze my account for thoughtfully answering questions on Yahoo! Answers, I saw a lot of questions asking about harassment.  There were those getting harassed who wanted to know what to do and those doing the harassment who wanted to know if they're going to get caught (typically after they posted the ad!).

I would always explain to the idiots doing the harassment how they would get caught, with exact technical details, and left them thinking the next knock on their door would be the police.  I like the thought of these losers living in fear, but I knew most police don't have the resources to pursue internet related cases.

However, in Missouri, there is a new law on the books and it looks like the police have it figured out.  This lady got caught and I hope they throw the book at her.

I'm starting to write a eBook titled "Buying and Selling on Craigslist" (wicked original, I know).  When the ebook is finished I'll give it away for free. 

The purpose of the eBook is to educate people on how use Craigslist effectively.  People new to Craigslist often have a lot of questions about the buying and selling process that go unanswered until they learn them from the school of hard knocks.  My hope is to create a more standard and safe way of buying and selling on Craigslist; a point of reference that both buyer and seller could point to as an ideal transaction process.

• Proper ad posting and ad replying etiquette.
• Handling payments and documentation for private sales.
• Weeding out scams and crazies.
• Basic how to information, step by step.
• Real world examples, I will buy and sell something on Craigslist while documenting every detail.

I plan to post fragments to the blog as they’re being developed to get feedback from visitors that can be used to make the eBook even better.

Gary Wolf is doing an excellent series of posts on Craigslist for the Wired Epicenter blog.  One thing not covered in this post about Craigslist's competition is the chicken and egg problem with classified websites.  No one is going to spend time and/or money to place an ad on a website that get's very little traffic.  No buyer are going to check a classified website with very few ads. 

I've tried kijiji.com once (dumbest name ever?).  There was one listing under "boats" in my area (we're surrounded by water here) and I've never gone back to Kijiji since.  Whereas, I look at all the boats on Craigslist daily and it never disappoints (yes, I'm a little boat crazy).

To change this chicken and egg problem a conventional approach would be to advertise.  However, the economics of free services (assuming any real competition to Craigslist would be free) means there is no revenue to pay for advertising.  This creates another catch 22, if you have no revenue you can't buy massive amounts of traffic and you can't earn revenue without traffic.  Of course, you can buy traffic with venture funding, but that's more of a gamble than anything.  Gary hint's that Oodle.com is a 20 million dollar gamble that failed.  I would have to agree, even with their integration with Facebook.

If anything, there are online classifieds that focus on a singular type of product.  AutoTrader.com and iBoats.com are two sites that I'll check if I'm looking for a car or boat.  AutoTrader.com has done a great job parleying their print classifieds online.  iBoats.com is a great boating community site with classifieds bolted on.  These are legitimate ways to get a classifieds site the ground in my view. 

Wired has a great online piece on Craigslist that gets deep into why Craigslist is the way it is.  I've followed Craigslist for a long time now, so there was nothing new that surprised me, but if you don't understand Craigslist's dynamics then this article is an eye opener. 

Here's a quote that I've known for some time:

"Think of any Web feature that has become popular in the past 10 years: Chances are craigslist has considered it and rejected it. If you try to build a third-party application designed to make craigslist work better, the management will almost certainly throw up technical roadblocks to shut you down."

I built NotifyWire.com knowing this was their attitude.  For every feature that get's added, I think, "OK, if I was Craig, what roadblock could I throw up to kill this?"  At the end of the day, there is really nothing Craigslist can do to shut down NotifyWire's Craigslist search tool.  Not to get too technical, but NotifyWire is essentially a fancy, distributed, RSS reader and web browser made specifically for Craigslist. 

Soon, NotifyWire.com will expand to include simple to use features to eliminate SPAM and SCAMs as you reply to ads or post your own.  Again, with these new features, I've also built it in a way to ensure we can't easily be shut down by Craig.

It's not that I want to be stand-off-ish with Craigslist, but this is the reality they've created.  On the plus side, they've created a market for someone willing to play this cat and mouse game. 

Today, a federal NY judge favored on the side of freedom.  He dismissed a $10 million dollar lawsuit against Craigslist brought by a shooting victim.  The shooter, who was crazy, had purchased the gun on Craigslist, thus Craigslist is at fault, right? 

If the gun was purchased out of the trunk of a Ford Taurus at a McDonald's parking lot, does that mean Ford and McDonald's should be sued?

I do feel sorry for the victim, but if this line of thinking wins the day, we'll be living like the North Koreans in a no time.

Here's the full story.