What happened to Mildsurprise and Priceslice? they are both down.
MildSurprise and PriceSlice?
LOz-bargainer on 23/09/2014 - 21:18
Comments
I was wondering about this myself, and had just logged in to ask this very question. I was wondering whether these sites had been shuttered due to excessive costs, or due to some other issue? Did PriceSlice give a reason other than "closed indefinitely"?
The reason I was looking into this stuff is because I've been working in my spare-time on a similar problem, and I've been running a private price-tracker for about an year now. Initially, it was tracking about 5 Aussie retailers, and recording price changes daily. Since about a month or two ago, I've added 5 more. I never opened it up because there were other substitutes like MildSurprise and PriceSlice already, and because my end goal wasn't to build a price-tracker. (but rather, to do some information extraction/data analytics stuff - a long, hard slog in my spare-time)
At the moment, I'm running my tracker on a private VM (both the crawler, and the webapp), and the performance has been acceptable - since the only user is me :-) The crawlers run once a day, and do their best to be very polite, and I haven't noticed any attempts to block them or anything.
If it benefits the community, I'm happy to move the site to a bigger server on Linode or Binary Lane or something (recommendations welcome), and make it available publicly. I've also not integrated features like watch item and price drop emails, but it should be possible to bolt that on fairly quickly. It's also possible to expose the current ReST API for getting min/max prices.
What do you guys think?
But that's basically how MildSurprise started. Just make sure you don't lose interest and 12 months down the track people posting question on OzBargain "where is raziel's price tracking site?" :)
Sure, that's a a fair point :-) However, I have been running this for an year now, and my experience has been that, since the initial investment in time, things have been pretty stable, requiring minimal active maintenance. Cost is also unlikely to be a factor, since I'm currently running everything on a pretty sad instance, and it's holding up fine. There are many reasons I'm getting away with such a small machine, among them, async concurrency, and not using a 64bit machine - it's all tried and tested stuff, so I have a pretty good idea how affordable it is.
Granted, that same configuration won't be able to handle tens of thousands of users, but that would arguably be a more pleasant problem to solve :-) If many people are deriving a lot of utility out of it, I'd be happy to put in a lot of effort. If not, since I'll be working long term on the crawler anyway, I am reasonably confident that I can keep it chugging along. In a worse case scenario, I can always turn the mantle over to someone else. As long as the site is constrained to harvesting a reasonable number of sites, I don't expect the costs to skyrocket.
Well that's a downer. Hopefully CCC can figure something out, it's a great service.
Surely there is a workaround, there usually always is. Esp. with tech - they can always screen cap and use OCR is Amazon just blocks everything and puts obstacles up.
Yes there's always a workaround. However it's more of effort you put in vs. the gain that you might get out — unfortunately online publishers aren't charity organisations with unlimited time.
I learnt about the change from this blog post from Pricenoia. Related Hacker News discussion and previous Pricenoia discussion on OzBargain. They contacted me earlier this year asking about replacing Camel x3 on OzBargain with their widget. Their embed-able charts look much better so I said "sure I'll have a look", but then somehow forgot about it. Now they are shutting it down…
I have not been tracking their availability, but emailed PriceSlice last week (as we used their API to pull the min/max price for some stores). Reply from PriceSlice:
While these price comparison sites provide great tools, they can take significant resource to run, as they "scrap" retailer's website multiple times a day to retrieve the price. They can be easily blocked by the retailers. Some have employed firewalls to block bot access. A lot of trouble for little gain I guess.
I heard Amazon is also blocking API access from price comparison sites. Not sure how long CamelCamelCamel is going to last.