A new website to quickly look up current supermarket deal

Associatedenveloped @ au.whatson.sale on 22/06/2015 - 17:00
Last edited 23/06/2015 - 11:03 by 1 other user

I just built this website, hoping it be useful:

http://au.whatson.sale/
(now closed from public access in response to replies)

It's designed to be small enough, contain only essential information, for convenice of skimming through the lists on a mobile phone.

I know mygroceries.com.au also have this information, but they often have "this week's market special is not yet there" message - so often to the extent that I skip it for Googling. I think they couldn't update it frequently because they duplicated the work (they have to manually update their list).

This new webiste uses Google's scripting technology. It automatically senses tightarse's current-week post and update itself to it, hence there is no extra weekly labour for me, and hence updates more frequently. I set it to update every 11:59pm.

You may wonder if I want to avoid duplicating the work why don't I contact mygroceries.com.au? Well I did last year wrote a long email to them on suggestions, but no reply. That kind of disheartened me.

P.S. I am not sure if I should be marked "Store representative" because that little site is not a store and it isn't selling anything nor ask for subscribers. It's just information aggregation.

Quote from "about" page:

This is a simple website.

Moving to a new country is not easy for me, and Australia is not a cheap
place:) as a result I am easily attracted to latest supermarket deals.
But the supermarkets don't always make it easy to browse.

This website fetches the latest supermarket deals from Google, with a
computer program - more specifically, "python script".

It took me 2 days to get the website sorted out. If it helps me, it is maybe
helpful for others too:) If you find the information useful, thank the respective
authors for each piece of information (source is at the bottom of each page).

Food & Grocery

Related Stores

au.whatson.sale

Comments

spackbace on 22/06/2015 - 17:02

It automatically senses tightarse's current-week post and update itself to it, hence there is no extra weekly labour for me, and hence updates more frequently. I set it to update every 11:59pm.

So it doesn't scrape Coles/Woolworths catalogue, it scrapes TA's posts on here? Cmon, thats just profiteering off someone else's hard work! I have put up a post before of a catalogue's specials and it aint easy!
- Associatedenveloped @ au.whatson.sale on 22/06/2015 - 17:05
  
  What should I do to make information easily available without offending TA? I already marked that information is from TA and thank him on the front page. There are only 3 sentances on the front page. I guess I am experiencing a culture shock. If I am profiteering I have to profit right?
  
  It's so easy to make mistakes here. I'll just close the website from public access.
  - spackbace on 22/06/2015 - 17:06
    
    +1
    
    What's wrong with what Ozbargain currently has, with a simple search for Coles/Woolworths deals?
    
    What's to say you don't add Google Ads later on when they hit-counter increases.
    
    Seems like you're banking on TA posting the deals for you.
    
    Not cool.
    - Associatedenveloped @ au.whatson.sale on 22/06/2015 - 17:08
      
      Okay okay, I closed the website. What do you think I should do now?
      
      What's wrong with what Ozbargain currently has, with a simple search
      for Coles/Woolworths deals?
      
      The main problem is with mobile phone:
      
      Go to website
      
      Find latest woolworths and coles deals
      
      Find the 1/2 deal post. It may not sit in the 1st page, and the one on 1st page may not be current (it's about the coming week).
      
      When you swtich away from the post you found, it has to be reloaded because webpage is too big. And usually this is the time when I am in the super market, so reloading isn't easy.
      
      If you are sure that I am the only one with this problem (that is, use-case invalid). I'll simple take this as a scripting language exercise and be over with it.
      
      On the advertisement part: With ads it loses the reason why it was there: to be fast and only provide quick essential information. I chose to run it on free-of-charge Google Sites because that way there will be no cost that I need to cover. I don't have the idea or plan to put an ad, but I guess that won't be believed anyway.
    - mooboy on 22/06/2015 - 17:31
      
      +2
      
      You closed it because just one other person found fault with it? Unless that person is TA himself, I'd wait for other feedback.
      
      As long as you have no plans to add ads, I'd just see this as another way of publicising TA's work - which TA may or may not have issue with.
    - spackbace on 22/06/2015 - 17:32
      
      @enveloped:
      
      If you are sure that I am the only one with this problem (that is, use-case invalid). I'll simple take this as a scripting language exercise and be over with it.
      
      Even if you're not the only person, the website you develop should scrape information from Coles/Woolworths directly, not this site. Whether there's intellectual property things involved, or not, as I said before you're backing your information from a certain member/certain post. How about asking that member if you can do it? That's the first thing you should look to do. Then ask Scotty if it's fine. That's second. Then make sure your script allows for another member posting the information instead of TA.
      
      Or be prepared before you walk into the supermarket?
    - Associatedenveloped @ au.whatson.sale on 22/06/2015 - 17:40
      
      @mooboy: I am really twitchy in the new country and new forum. It's so easy to say something wrong and people start to blame (had a few bad experiences on this site besides good ones). I know the posting guide say do skin thickening but I lack the culture background to judge if I look stupid or brave by fighting. Besides, although this site is more or less built for fun, I am starting a serious new online store (not this one we are discussing) and that one is for profit (which I won't shutdown because of one or two bad comments). 2 days of work on a for-fun project doesn't worth the potential bad credit to my ID that is linked to the store, better be careful. The scrutiny of this forum is unmatched with my previous experience. But you made a point, I already wrote a message to TA and I perhaps follow up to Scott later if the site is to open. Now it seems to me this is a tiny project not worth the trouble.
      
      How stupid I thought I would receive "thankyou" messages …
    - Associatedenveloped @ au.whatson.sale on 22/06/2015 - 17:50
      
      @Spackbace: Other parts of your reply I take it as a learning experience (= accepted). I reply to this part:
      
      Or be prepared before you walk into the supermarket?
      
      It's not the way it should work in the modern world. In other countries people don't have time to prepare e.g. shoppping list. I am afraid even in Australia a lot of for-convenience stuff are loved by people.
      
      The reason I only source from TA and only from ozbargain.com.au is because I worked on it as a weekend's for-fun project and chose a technical problem small enough to solve in a weekend. I thought if people likes it I'll ask what people need from it. It's kind of silly to think of a long plan for a 150-line program when I am not sure if anyone would actually use such a thing. I thought it interesting to observe its usage, a kind of social experiment, a way to test if my perception of users' need is correct.
    - gooddealmate on 22/06/2015 - 19:40
      
      +2
      
      @enveloped:
      
      I know the posting guide say do skin thickening but I lack the culture background to judge if I look stupid or brave by fighting.
      
      Don't worry, you don't look stupid. I admire your enthusiasm. You have some great ideas you've put across on this forum, I think you just need to think them out a little better.
      
      I'm sure TA wouldn't take offence at your project either, we all like to see something that makes finding a bargain easier, so don't give up, keep 'em coming. :)
dylanando on 22/06/2015 - 21:09

Ahh wheres my site? Put it back up. Not everyone has to like it. Put your shell back on and make it public again.
Would love to hear your other ideas and see what else you bring to the table. I want bargains!
PVA on 22/06/2015 - 21:30

you closed access after 5 minutes. give others at least look so we can all comment.
- Associatedenveloped @ au.whatson.sale on 22/06/2015 - 23:05
  
  Okay I put it back. I actually messaged TA. He(she?) didn't fret about it and neither commented, simply "thanks letting me know". I think he(she) simply isn't interested about it:)
Davo1111 on 22/06/2015 - 23:11

Honestly, i didn't like it. Just seemed like plagiarism to me.
- Associatedenveloped @ au.whatson.sale on 22/06/2015 - 23:31
  
  Thanks for the new English word. The point being small (15kb) and 1-click to the list. I didn't care about name and messaged TA to giving the site to him/her so this become a true real probe of user needs - TA didn't bother responding yet.
  
  I thought about it. I repeated what is provided by mygrocerious.com.au's "this week market special" after they (almost) stopped updating for months. I first thought it was the cost issue, so the solution is to automate it. But on a second thought, if this service is really needed, they wouldn't stop providing it on cost ground. It seems to me that it's not the cost, but on such needs. They provided an unneeded service. That also explained why they didn't reply my email last year. It could be that I am one of the few users of that service and the reason I found it useful is that I came from a different culture background.
  
  Given another chance I wouldn't just start building it, instead, I would first make a post asking if anyone miss their services. It was so apparent to me that theirs was a valuable needed service, that missed to check others.
  - spackbace on 22/06/2015 - 23:33
    
    so my solution is cost-zero & automatic.
    
    Because My Groceries accessed the catalogue with their own software.
    
    Again, what's your fall back plan if TA doesn't post?
    - Associatedenveloped @ au.whatson.sale on 22/06/2015 - 23:35
      
      -2
      
      I have no plan. I did it in an weekend to check how much people like it. The idea was, if the service is there but mobile access is zero, I won't do anything (the site has no hosting cost). If there are some mobile access, I'll make another post asking what to do next, or maybe I'll find some use cases by looking at mobile access data (e.g. if people look at the list while choosing products from the self like I did).
      
      If people use it there will be discussion & solutions when TA stop posting. A needed project doesn't die easily. Most websites die because no one care to use them.
      
      The motivation to solve it came to me on Thursday, so I took the domain that day and planned to make it over the weekend. Today is Monday. You can tell that there are no time for big evil plan by looking at the date difference between domain registration and site online date. I tends to see (or thought I see) a need then start to do it right away. My other (main) online store opened only a few days after I posted the idea.
  - Davo1111 on 22/06/2015 - 23:44
    
    i just don't get the purpose. The only difference is that someone can go to your website, instead of going to ozbargain on a Sunday. If TA created a special "tag" in every post, you could subscribe via RSS/email and your website would be completely redundant.
    
    What you've done is pretty much the definition of plagiarism, zero modification. If you had done this from a popular website, you probably would have received a cease and desist letter or a DMCA takedown notice.
    - Associatedenveloped @ au.whatson.sale on 22/06/2015 - 23:57
      
      The only difference is that someone can go to your website,
      instead of going to ozbargain on a Sunday.
      
      So is the service I missed (mygroceries.com.au/supermarket_specials). They are also around 17kb.
      
      What you've done is pretty much the definition of plagiarism,
      zero modification.
      
      Zero modification is intended and I noted that I don't own the information on every page of a 4-page only website. 3 of which links back to ozbargain.com.au, since the begining. One to tightarse's tag page, two others to individual posts. Normally site owners love back-links. It's not about owning the right to a master piece, it's about accessing it fast from mobile on the way to the supermarket. I made no attempt to pretend the information is not copied, and explained how I copy it in the top post on this thread.
      
      TA already have own taged web page, but it contain too many posts (many not applicable in the on-the-way-to-supermarket case, e.g. future-deals) and in each too much page content (slow to load). On mobile phone I try to use mygroceries.com.au before trying on ozbargain, even knowing the information is same/better on ozbargain. Their cease of timely updates is a problem.
      
      I don't need a DMCA take down notice - I closed it in 5 minutes on the first complain (not even from TA).
      
      There are little sense in trying more. I'll close it soon (tomorrow?), but leave it open for now just in case someone would comment on the use case side. I am feeling that thare are time-poor and bargain-enamoured but only who are on both sides would need it - and ozbargain community is more on the bargain-enamoured side.
    - spackbace on 23/06/2015 - 00:27
      
      If TA created a special "tag" in every post, you could subscribe via RSS/email and your website would be completely redundant.
      
      That's an interesting idea. If it was just tagged with 'Coles50%' or 'WW50%' it would be very easy to rss subscribe. Hmm
    - Associatedenveloped @ au.whatson.sale on 23/06/2015 - 00:29
      
      @Spackbace: Yes, with special mark-away future-deals and post-content (but not discussion) in RSS itself (perhaps the latter is already done. Perhaps can be really quick to access. I don't use rss myself, should to get used to it too. If I get familiar with RSS on mobiles in a month I may think the old idea silly and regret learning it the hard way.
Ihhdle on 23/06/2015 - 09:02

I like it. Very handy and simple - just what you need when you're standing in the supermarket. Hope you get to leave it up.
Lurker on 23/06/2015 - 23:30

Thanks , please leave it on if tightarse or other data providers have no problems with it.

Would love to learn how to scrape websites from you one day.
- Associatedenveloped @ au.whatson.sale on 24/06/2015 - 14:01
  
  +2
  
  The technology involved in this tiny project is menial - lots of steps for setup but only a little programming skill needed. The source code is here, use it whatever way you like:
  
  http://pastebin.com/iEYEhSN6
  
  in case you want to run the code yourself:
  
  register and set up a Google Sites.
  
  replace 'sitename' in the source with the name of your site.
  
  Register Google API, request a installable client and obtain its client ID and secret (you may need to create a project first - I did).
  
  Replace both in the source.
  
  Install python-gdata module.
  
  Run the code and authorize this application to edit your Sites. This gets you ~/.gdata-storage which is the authorization token. (I only tested it on Linux)
  
  Then you can run the code. Notice your site should have the 2 named pages that the source code attempts to update.
  
  if you like to scrape from more than one sites and stuff them into a Google Site - IMHO google.com is the biggest website that does this:) - for each site you need to create a google custom search (and obtain a key to be used in source code), and google limits you to about 5 sites. You have the option to close your site from public access.
  
  Now I can't get reply from tightarse yet. I am afraid this is too tiny for his attention yet.
  - Lurker on 13/07/2015 - 10:20
    
    Thanks a million.
    
    You are an angel.
    
    I am learning Python now and one of my goals is to use it to scrap websites.
    
    Your code will be a great source of learning Python for me.
    - Associatedenveloped @ au.whatson.sale on 14/07/2015 - 22:14
      
      +1
      
      If you are studying, perhaps you can start from this simpler one I wrote, which I put here:
      
      https://bitbucket.org/cookthis/scraper/src/93e8f87e1dc365a11…
      
      Run the recipes.py (recipes.tsv has to be in the same directory/folder)
      
      Then you can see it reads the recipe, and get each ingradient from woolworths website using json, then calculate the total cost of the recipe:)
      
      I am not a star programmer, just like to share.
    - Lurker on 15/07/2015 - 19:29
      
      @enveloped:
      
      Thanks for sharing , enveloped.
      
      Unfortunately it says:
      
      "Access denied"
      
      May be you can make it accessible or put it up github?
      
      Thanks a million.
    - Associatedenveloped @ au.whatson.sale on 15/07/2015 - 22:44
      
      You are right. Eventually I'll do everything in github, here is it for now:
      
      the python code:
      http://pastebin.com/KhicnY0X
      
      recipes.tsv
      http://pastebin.com/g1mnpbcS
      
      recipes.tsv has to be in the same directory as the python code. On my computer it produces a shopping list like this:
      
      $ ./recipes.py
      Qty__Price__Each___Name
      3____1.5____200g____Fresh_Banana_Lady_Finger
      1____1.2____1000g___The_Odd_Bunch_Carrot
      1____4.2____215g____Maille_Mustard_Dijon
      
      Starting from here you can do interesting things like finding recipes on the Internet and choose to cook the recipe that is cheap due to specials.
      
      I met a former manager from Coles who said that the special ½ priced products are indeed too cheap, that all they hope for is customers to buy other expensive things together with them. So I guess automatical 'cheap' recipe scraping is a bit overdoing it:)
TrolleySaver on 24/06/2015 - 17:46

+1

Not as simple as yours enveloped, but our database for Coles and Woolworths is available here: www.trolleysaver.net

It contains all catalogue specials information and is searchable.

Please excuse the overall ugliness - still working on incorporating it into our main website (hence the .net)
- PVA on 24/06/2015 - 17:53
  
  +1
  
  not bad. I notice double entries a lot when I search (say for "coffee"
  would be good if it worked out the unit price for all items so we could compare.
  - TrolleySaver on 24/06/2015 - 22:27
    
    Unit prices are hard for some items because they list a range of product sizes for the same price (e.g. Starburst Medium Bags 104g - 180g for $1.31….unit price only able to be calculated on an individual product basis)
- Associatedenveloped @ au.whatson.sale on 24/06/2015 - 18:30
  
  +1
  
  Hi. I have a few suggestions to start with:
  
  Whenver I want to work on a project, I look at use cases first. Problem being I am not sure if the cases are valid.
  
  There are a few use cases when users need that trolley database. Correct me with your experience please!
  
  1) when people are interested in the current weeks's special for planning.
  (this is the case where ozbargain is doing it already 100% right)
  
  2) when people's favourate item is added to weekly special and they want to get notified. Woolworths and Coles themselves have this information and they can easily notify their customers. Why they don't? Perhaps they prefer customers to chance them and attribute their success to frequenting their shops hence visit them more often. mygroceries.com.au allow users to type their favourates for them to notify but lack maintentance, and buggy (it misses instead of giving false positives).
  
  3) when they are already in supermarket (or a few miutes away) and want to know what to buy.
  
  4) when they are in a supermarket looking at an item and want to know if they should buy.
  
  I only wanted to play with the 3rd use-case. Since you have so much commitment maybe you should cover all use cases. Here is a few points I think for you:
  
  if you want to solve case-3, it has to be either small or fast to load. Drop non-essential information or show them only when user clicked. e.g. remove "frequency" and replace it with a star, so items normally not discounted will be star-ed. It better not be a table, but a list with expandable info. (I used a link-back-to-ozbargin instead of expanding more info).
  
  if you want to solve case-4, things become really interesting:) You either have to offer user to scan the bar-code (there are existing apps but didn't factor in weekly sales info), or, ask them to speak the name (spinach), because they won't be standing typing and look silly. Then, you have to not only tell customers if the discount or price good, but also look up location information, tell users whether or not there is an aldi with the same-named product within 300m. Such an app would be fun to make. But I didn't consult users if this is what they want, and a new settler can make a lot of wrong assumptions of user's needs.
  
  Anyway, good luck with your work!
  
  about 'ugly': usability problems have higher priorities. And there is one usability bug I noticed: there are double-scrollbars. So user who scrolled the outer scrollbar, will lose their inner scrollbar unless they first roll back to the frame where the innerscollbar's handle appears (or that they do a long-haul with mouse wheel). double-scroll bars are especially bad for handhelds.
  - TrolleySaver on 24/06/2015 - 22:26
    
    Thanks for the feedback! This online version of the database is very early stage - we're focusing on the app at the moment, which we are completely revamping and actually addresses most of your feedback.
    
    I think your solution for 4 is interesting. I like the sound of scanning the barcode - nice and easy, but same problem can be solved using our search function. Perhaps we'll look into the barcode scanning at a later stage.
    
    Hoping to have some time to optimise these screens for mobile soon.

A new website to quickly look up current supermarket deal

Related Stores

Comments

New Forum Topics