After a week of craziness, the monitoring service is back online.  Basically, there has been a lot of weird things happening with NotifyWire because of the Adobe Air framework upgrade that's been foisted on us.  While I love developing applications on the cross platform Air framework, it lacks one major feature, side by side execution.

Side by side execution allows people to have multiple version of a framework installed and allows applications to execute on the version which they were designed for.  Adobe Air on the other hand automatically upgrades the framework on the user's computer and forces existing Air applications to use the new version.  Right or wrong the new version has some quirks, or the old one does and the new one fixed them, either way they don't behave themselves in a few key areas.  

I'm actually still working to fix problems a few users are experiencing, but at least at this point I believe the monitoring engine is all set.

If you believe you're not getting alerts that you should be getting, I want to hear from you!

In an effort to update the monitoring engine and make it faster, I accidentally added a major bug (that only happens in certain conditions) but causes an endless loop in the program...very very bad stuff.

No one has reported the problem yet, but I've noticed it and I'm sure other might have and just not reported it.

Therefore, it's vitally important to follow these instructions once, until I update the software.  

All you have to do is click on the Help tab and click the Exit link at the bottom right of the page.  Then restart NotifyWire.  The monitoring engine will no longer be running, but at least the endless loop problem won't occur.  

I'm really sorry for his problem and will release an update as soon as possible.

Thanks,

Ian Drake

I'm in the process of testing changes to the our monitoring engine, specifically the NotifyWire search engine. You see, when monitoring Craigslist for your searches, NotifyWire doesn't actually run each individual search on Craigslist. Why? Well two reasons. First, it would require a lot more hits against Craigslist's servers, and second, it would actually be slower (there's a long story there).

Instead, NotifyWire just get's all the posts and runs your searches using our own search engine. This search engine was designed to mimic the engine used by Craigslist, so the results should match. I thought it was all working OK, but have had some reports recently about missing alerts...which I take very seriously.

I've probably spent 40-50 hours this week working on a test harness and fixing issues. The test harness uses real searches from NotifyWire users and runs the search against Craigslist. I then run the same results through the NotifyWire search engine, expecting each one evaluate to a positive match. However, on the first try I calculated a 10% mismatch. I also created a test for false positives that generated a 2% mismatch. Obviously this was unacceptable, so I got to work.

The results were so off, that I'm now wondering if Craigslist had changed there search engine because it was matching results in a much more flexible manner and some logical conditions that were broken now seemed to be fixed. For instance, you couldn't group negative terms together in parentheses before, but that works now. Here's another interesting tidbit - Craigslist's displays ads as they were written, but behind the scenes indexes an automatically spell checked version of the ad. I found multiple instances where Craigslist returned results that didn't have a required term properly spelled, so I'm thinking they don't do spell checking on the fly for each search, but index the spell checked results will all HTML tags removed.

I tried to run an open source spell checker against the ads before searching them, but the results were mixed. Out of ten cases found, the spell checker only got the right word once. For the amount of processing required, it wasn't worth it, so I pulled that out. In the end, the NotifyWire search engine now has a >.1% false negative rate and a 0% false positive rate. The remaining false negatives (should have matched but didn't) are due to spelling and the fact that one user's job search was based on word "compensation" that appears in the very bottom of a job ad, outside of the user provided content. Craigslist only provides the user generated content in their RSS, so the text is never found, but Craigslist includes it in their own search index.

I'm really happy with the results of re-writing the NotifyWire search engine.  I expect to release the new version on Sunday. Also in this new version will be navigation hotkeys, something that's been missing for far too long and really makes the NotifyWire application even more fun to use.

Last week I ran a design contest on 99Designs.com for two banner ads.  I was hoping to get a great design that would illustrate the benefits of NotifyWire and not just be another ad, flashing click me!!!

There weren't too many entries, perhaps because I wasn't offering a ton of money to the winner, but I sure did get lucky with this designer.  His entry was blew my mind, because it encapsulates the exact problem NotifyWire solves in just a few short seconds.

Not only is this ad great, but the guy in it kinda looks like me!

I'm always amazed at the clever people who use NotifyWire.  Some people use it to save a ton of money, while others use it to make money.  Users primarily search for cars, trucks, boats, bikes, electronics (like cell phones and computers), houses, and apartments. 

Some customers are experts in certain products and are able to buy cheap items and resell them for a profit.  In some cases, this requires extensive knowledge to repair damaged items and sometimes customers just look for sellers dumping their belongings in an attempt to satisfy their spouse or meet their mortgage payment.

Actually, that's one of the reasons I started NotifyWire.  I was the guy posting ads to dump things my wife wanted gone.  We sold some really great stuff for very little cash because we just needed room and needed it fast.  When I posted a $400 leather chair for $50, I was amazed how fast people responded.  I had a buyer in 15 minutes.

After that, it occurred to me that you had to get really lucky to get great deals on Craigslist.  NotifyWire was built to remove luck from the equation.  It was built so you could respond before that 15 minute threshold.  In fact, I recently called a guy about a vintage surfboard (that's what I monitor Craigslist for) and he was stunned.  

He said, "I just posted that like two minutes ago."  I know.

I just finished making a big change in the way NotifyWire monitors Craigslist and the results are fantastic.  This change will help NotifyWire monitor more of Craigslist, more efficiently, and more frequently, than ever before.  In fact, the new version will reduce the number of hits against Craigslist by 97% (when monitoring at the same frequency).

I have some a breakthrough for speeding up your BIG Craigslist searches coming too.  You might notice a few NotifyWire releases during the next week as I start to roll out this feature silently.  It won't be immediately available, but I will "turn it on" for certain customers to test.  When everything looks good, I'l enable the new search feature for everyone.  I expect to see searches run about 20x faster...not 20%, but 20X!

The new server seems to be running great.  However, because NotifyWire changed its IP address, some ISPs (ATT, Comcast, AOL, SBC) are blocking emails from NotifyWire.com  I've been working to get the SPF record right for the domain and working with these ISP to confirm NotifyWire's mail server is on the up and up, but it may take some time.

If you are with one of these providers and your email alerts are important to you, you might want to consider adding another email account from gmail or yahoo to your NotifyWire.com account and adding it to your searches.  I don't believe this will be a long term problem, but I'm not sure how long it will take with some ISPs.

Update: I haven't been getting as many bounce back emails lately.  I think that means the SPF record is right and has replicated across the Internet.

Update 2: AOL seems to be the only ISP still blocking NotifyWire alerts.

If you're seeing this blog post, you're seeing NotifyWire running on it's new server.  The NotifyWire application has been update with several fixes and new features.  Here's a brief outline:

• Added new Alert tab to display and manage your alerts right from the app
• Added a new Settings tab to add/remove contact info, select an email provider for replying to ads, and undeleting your deleted ads
• Fixed a bug when adding a new contact
• Changed how paging is done because it was buggy on some computers.  You can no longer hold the left mouse button down to continuiously scroll through pages.  Instead you have to click for each page.
• Various small bug fixes

One big change that has to do with the website.  I've removed the online functionality to view your alerts and saved ads.  Note, you can still view individual alerts using the alert code sent to you via SMS and email, but that's it.  

Why did I remove a feature?  Well, for one, no one was using it.  Secondly, it was confusing a lot of people including a PHD who insisted that, if he can't figure it out, then I'm doing it wrong. ;)  Now that you can fully manage your alerts through the NotifyWire app, there's really no reason to keep the same functionality on the website.

While I'm really excited about the new version, I realize that there may be a few bumps with the new server and software version.  Please report any problems you notice to Support@NotifyWire.com.

The new NotifyWire.com server is all setup and is now undergoing testing.  I'm excited about the performance improvements and I think uptime will be vastly improved because I can directly monitor loads and better optimize where necessary.

If testing goes well, I'll be switching to the new server in the middle of the night tomorrow.  

I'm not able to release as fast as I hoped, but I've made a lot of progress in the last few days.  To get the new VPS hosting solution running I had to setup a DNS server, something I'm not familiar with, but it looks like everything is working now.

The next step is to setup the mail and ftp servers.  Then I'll be ready to install the new version of NotifyWire which will have some very cool new features.  I wanted to get all this done by last Sunday night, but I'm afraid I didn't know how much I didn't know about setting up a hosting server.

In retrospect, I wish this was something I had done earlier.  Running a successful website/webservice necessitates acquiring these skills and I hate to be learning on the job at such a critical time.  Oh well, nothing a few long nights can't take care of.