[Linux] Free - Train your own ChatGPT 80% Faster + 50% less memory @ Unsloth.Ai

AssociatedLmoonshotai @ unsloth.ai on 01/12/2023 - 19:31
Last edited 02/12/2023 - 11:39 by 2 other users

Moved to Forum: Original Link

Hey there fellow OzBargainers! 🦥

Today, we've launched a new open source package called 'Unsloth' on Github which reduces the time it takes to train your own personal ChatGPT by 80%. It also reduces your memory usage by 50% + much more. PS. Originally the package was launched as paid but we just ended up deciding to open-source it.

You can try it right now for Free on Github. The installation instructions will all be on the page.

If you're new to language models or AI etc., no worries, we're more than happy to help a fellow OzBargainer on our Discord server

See here if you would like to read our blog about Unsloth

We're Moonshot, an AI startup based in Sydney, Australia.
Thank you

Free Software/Websites

Related Stores

unsloth.ai
unsloth.ai

Comments

  • Jimothy Wongingtons on 01/12/2023 - 19:32
    +4

    How long till can train it into yet another nsfw ai chat bot?

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 19:33
      +3

      Hahahah well technically you could, the pacakage is open-source. You could do whatever you want with it.

  • Winny182 on 01/12/2023 - 19:36
    +5

    Just curious what the actual deal here is? Just seems like an advertisement.

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 19:39
      +13

      Great question! The package is completely open-source and free to use. There's literally no strings/costs attached. Sure, we are posting it on OzBargain to get more people to use the package but that's about it really.

      • Bedgrub on 01/12/2023 - 21:21
        +8

        I saw the post on HN and noticed it didn’t get any traction.

        I’m all for promoting OSS, and I think that OzB’s community could benefit from a lot of well developed alternatives.

        However, the LLM and AI market is still working itself out, and there’s a lot of charlatans.

        You should be clearer about your products commercial goals. Sure, the project on GH is Apache 2.0 licensed, but you also seem to be angling towards some sort of subscription model?

        Without being clearer about the business proposition, you run the risk of looking like another repackaged LLM.

        • pleasehelpme on 01/12/2023 - 22:00
          +2

          Just wanna ask what the meaning of HN is please.

        • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 22:04
          +5

          Thanks for the comment!

          The OSS code on Github allows you to train LLMs 5x faster and use 50% less memory for free - no strings attached - our Google Colab and Kaggle notebooks which are shared on the Github page show how you can train Llama faster for free 5x faster by just using your Google account.

          We do have a paid version, but that makes training further faster to 30x, or 97% faster. We never intended to monetize Unsloth, but our old project https://github.com/danielhanchen/hyperlearn made machine learning algorithms 2000x faster, but big corporations took advantage of it and other startups repackaged it without even crediting us since we open sourced all the code. Our goal isn't to try to charge for usage - the OSS version is fully free to use and download. Only once you want massive speed improvements then you might have to pay.

          For the Pro version, it'll be extremely cheap maybe the price of a game - but we haven't decided yet on pricing since we're doing is all on the go.

          Btw, we serve a completely different purpose from a 'repackaged LLM' or a LLM in general.

          • dingbated on 01/12/2023 - 22:49
            +4

            @moonshotai: Huh, you hurt my brain if you didn't want corporations to use the code period or without credit, all you needed to do was use a different license. Why are you playing these strange games.

            It's OVER 9000!

            • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 23:24
              +3

              @dingbated: If only it was ever that easy. By using the AGPL3 license which specifically forbids any company from using it will not solve any problem. Sure at the start, company's cannot use it, but coporations will later hire engineers to extract and monitor our code bit by bit and learn all of our tips and tricks (which we literally spent years working on) and then replicate it somehow and then use it in their own platforms for their own profit.

              This has actually happened to many open-source packages in the past with this license and it's been known to be a hot topic (e.g. Amazon stealing and learning from mongoDB which was in the news). And, after all, the end goal of a company/startup is to earn profit.

        • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 17:41

          Btw just an update but we actually managed to get #1 on HackerNews https://news.ycombinator.com/item?id=38487199

          • Bedgrub on 02/12/2023 - 22:45

            @moonshotai: Well done - there’s a lot of tough questions in that thread but you’ve done a bang up job responding to them. The general response seems to be one of disbelief, but if tour offering truly matches the claims - and if it is difficult to replicate - then it sounds like a compelling proposition that is worthy of backing.

            Good luck!

            • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 23:17
              +1

              @Bedgrub: Thank you Bedgrub for your kind comments!

              Hope you have a lovely Christmas!

    • HomeAlone on 01/12/2023 - 19:56
      +12

      https://www.ozbargain.com.au/wiki/help:deal_posting_guidelin…

      Products that are temporarily free or have become permanently free qualify as deals.

      • mun4 on 01/12/2023 - 23:36
        +2

        Who in their right business mind would launch a paid product for a few days but then suddenly decided to open-source it forever? Give me a break.

        • ilovefullprice on 02/12/2023 - 10:52
          +1

          It can be in the fast moving tech space especially when you’re dealing with novel technologies still sorting itself out in a messy landscape.

          • mun4 on 02/12/2023 - 11:56

            @ilovefullprice: No, it has nothing to do with the pace of the sector. If the business model was to sell the software or put users on subscriptions, it'd take way more than a few days to decide to pivot to an OSS model. The domain was registered on Nov 27 and they pivoted less than a week in? Either the team behind are extraordinarily naive or they're making up stories to market their product on OZB.

  • Pikaloo on 01/12/2023 - 20:11
    +1

    Does it work with this deal? https://www.ozbargain.com.au/node/814885

    I'm trying to compete with OpenAI.

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 21:47
      +1

      Hi there, the package only works with Linux/Ubuntu and NVIDIA graphics cards.

      • Bedgrub on 02/12/2023 - 07:05
        +1

        Have you considered porting it to M-series? There’s a sizeable portion of the market right there.

  • Tollery on 01/12/2023 - 20:17

    Today, we've launched a new open source package called 'Unsloth' on Github which reduces the time it takes to train your own personal ChatGPT by 80%

    This is a AI chat bot that we have to install and train ourself or do we already need to have one?

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 21:52

      Hi there! You can get free open source AI chat bots via FaceBook's LLama etc. and then you can train your own language model/data using our package.

      • Tollery on 01/12/2023 - 22:05
        +1

        AI chat bots via FaceBook's LLama

        That's the part that scares me. When facebook gets involved, it's all about data harvesting.

        • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 23:27
          +3

          Well when you're using the LLama model it's not connected to FaceBook in any way. It's open source and constantly used/monitored by thousands lots of engineers so if they something about data harvesting, they'll be sure to know by now. But ofc, noone is 100% sure what Facebook is doing with their open-source models.

        • Bedgrub on 02/12/2023 - 07:07

          They just built and release the model - you run it offline.

  • cucumber on 01/12/2023 - 20:28
    +1

    How does this compare to GPT4All?

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 21:51
      +3

      GPT4All only allows running. We allow training and running. :)

      • cucumber on 02/12/2023 - 12:29

        Ahh I thought GPT4All allows training too… Or was that just fine-tuning?

  • lachlantula on 01/12/2023 - 21:02

    this seems like a cool good value product but i can't help but feel like this post would be more suited to product hunt or smth

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 21:48
      +10

      I agree, we haven't posted it on Product Hunt but we posted it here first because I use OzBargain everyday and we're Australian so we thought it was the perfect place to post it.

  • TommyGun on 01/12/2023 - 21:12
    +1

    We'll done @moonshotai. Appreciate the effort, I've been meaning to give something a go via Akash GPU leasing.

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 21:48

      Thank you! Have fun with your training! :)

  • yzh1999 on 01/12/2023 - 21:36
    +1

    So essentially you are promoting a QLoRa implementation?

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 21:50
      +2

      Kinda? We essentially make open-source language models 80% faster and 50% less memory use via the QLoRa method.

  • Zabuza on 01/12/2023 - 22:05

    how is this different to custom GPTs? Sorry bit confused how it makes it faster, are you just offering more compute power?

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 22:07
      +1

      Custom GPTs is not training. We do finetuning which we increase the accuracy and speed of. And, by using our package, it also requires a basis for you to train on (e.g. data).

  • dealmaker on 01/12/2023 - 23:33
    +1

    Wish you all the best, great share.
    Hope you are able to monetize your hard work. That's where most businesses and talents fail unfortunately.

    • Associatedmoonshotai @ unsloth.ai on 01/12/2023 - 23:40
      +1

      Hi there thank you so much for this comment. It really means a lot to us and we really appreciate it! Hope you have a lovely Christmas! :)

  • BoltThrower on 01/12/2023 - 23:53

    Interesting. Is this part of a decentralized collaborative AI training?

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 01:00
      +1

      No, this trains on your own local graphics card/PC. So you don't even need the internet to train technically

  • SpaceNinja on 02/12/2023 - 00:11

    Can you please tell me some usecases of why one could train own GPTs? Thanks

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 00:58
      +2

      Hi there. There are sooo many use cases! You can boost productivity mostly, make more personalized bots (e.g. bot specifically for recipes) or just do it for fun. There are a lot of specific dataset or things which the OG Chat GPT does not have or offer. If I wanted to ask ChatGPT about how to complete a specific level in a game, Chat GPT won't be able to respond properly, but by training your own GPT, you can allow audiences to receive much better and accurate answers.

      Hope this helps

  • PopCounty on 02/12/2023 - 02:56

    Unsloth and Hyperlearn; beautiful names. The web site is so soothing to browse to. Some really good color choices. And all your responses have been so good.

    You are in for business mate. Wishing you all the best !!!

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 13:16

      Thank you so much! Means a lot of us.

      I'm actually the designer of Unsloth so it's really nice to see someone complimenting it ahahah!

      Have a lovely day!

  • Gibmefreestuff on 02/12/2023 - 04:10
    +1

    Just to confirm, this wouldn't work on macbooks correct? (M1)

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 13:16

      No it wouldn't sorry. But it will work on google colab, kaggle etc.

      We're working on compatibility issues.

  • ibfragalot on 02/12/2023 - 06:12

    Great work! I fully support Aussie innovation in the AI space, best of luck with your venture.

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 13:16

      Thank you so much!

      Have a lovely day!

  • GreenRomeo on 02/12/2023 - 07:39

    I’m just curious how this will play out. Assuming you can create a more customised model than OpenAI’s GPT by training your own data, how could it be meaningfully better than the general services than ChatGPT? In other words, unless you have a large and specialised set of private data and have access to huge computing resources, I don’t see how one can compete with giants like OpenAI, just like I didn’t see how smaller search engines could compete with Google, even when they just tried to focus on a narrow domain such as searching for a certain language or area. Happy to be proven wrong though!

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 13:19
      +1

      We aren't competing with OpenAI, we essentially make training for people's own chat bot's much faster.

      And yes, it will be more meaningful because ChatGPT cannot answer a lot of questions and by training your own, users will get more targeted, accurate and better answers. Technically you don't need to have 'private' data. There's A LOT of free data out there which a lot of people don't know about (Even engineer's and data scientists) but you need to take the time to find them and collate them into a usable dataset. Even OpenAI doesn't have all the datasets as it takes a lot of time and money to collect them all.

    • SummerZero on 04/12/2023 - 12:54

      It would probably have to be quite niche to succeed.
      e.g. I remember using a search engine called 'social sciences research network' to find papers on particular topics while at university. At the time it was a better way to search through research papers than google and the university's internal search engines.

  • ninnypoop on 02/12/2023 - 10:09

    Let's say I started a uni course. Could the Ai bot 'start with me' and then learn as we go. Say, I feed basic theory in. The Ai bot takes that and learns above and beyond? I then learn from Ai along the way?

    I'm a novice to this technology, so maybe I'm thinking about it wrong.

    I'm done with uni studies and now lecture. I'm intrigued how Ai will change uni for our students (we see this already with students utilising chatgpt in assignments - albeit, most of them get caught out for cheating and this is becoming easier to recognise with Ai tools that recognise the work of an AI). There's also talk to make assignments Ai friendly - can't beat it might as well utilise it. Especially, if the use of Ai will become common practice in the future. Of course, you also design assessments that test the students without Ai (labs etc.).

    • lookingforadeal7 on 02/12/2023 - 11:11

      Not sure about your general question but as far as students being caught for cheating with ChatGPT that's on them for not being smart enough to go undetected. At least when I did my whole degree it was more of a challenge of how to plagiarise without being detected in assignments then anything else. Maybe Uni is designed in this way on purpose, learning to regurgitate information as if it was your own thought or research.

      Realistically AI is just another software that is able to be used and although there's a grey area now there's not a possibility we won't progress to the point it is allowed in Uni to assist students just as any other tool would be.

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 13:22
      +1

      There are limits to how many prompts you can do on a chat bot. This goes for Chat GPT as well so there's always a limit for how much a session can learn. But yes, technically if you want to complete an assignment, a AI bot is able to learn as you go as long as it fits within the amount of prompts.

  • ilovefullprice on 02/12/2023 - 12:01

    Nice post OP, upvoted.

    From quickly browsing your GitHub, you’re not using vLLM? Or is this in your paid tier.

    As a fellow Aussie, hoping you can help with a noob question. Do you have an idea how to send data in batches to an LLM for inferencing when working on a single GPU? For eg. I have a spreadsheet where I wish to get inferencing from the LLM for each row in the spreadsheet. To optimize GPU usage, rather than sending each row individually to the LLM one at a time, is there a way to send batches of rows to the LLM to get inferencing for each row?

    • Associatedmoonshotai @ unsloth.ai on 02/12/2023 - 16:16

      Hi there thank you so much for the positive post! :)

      If you want any help you can join our Discord server

      We're more than happy to help you there! (I'm just the social media/designer guy so I don't know much about etc.)

      Have a lovely day!

      • ilovefullprice on 02/12/2023 - 18:24
        +1

        Hey, no probs mate. The questions I asked was quite technical anyways so I don’t expect most people would know.

        Thanks for replying tho. I have given your GitHub repo a star to increase visibility. Aussie startups supporting each other is how local startups grow.

  • Geoff-bargain on 05/12/2023 - 12:41

    Not sure if you are still monitoring this but I was planning to try with my old 1070 - will it just not work at all or not as efficiently?

    • Associatedmoonshotai @ unsloth.ai on 06/12/2023 - 01:49

      No sorry. The minimum requirements are RTX 20 - something. :(

      • Geoff-bargain on 06/12/2023 - 09:15

        might be my excuse to upgrade then :)

  • Dbdpcg on 08/12/2023 - 22:03

    This looks pretty cool! I'm more of a ML-OPS engineer so not completely versed in what exactly this is doing. I wonder if this would let me train/fine-tune an off the shelf local LLM on a GitHub repo?

Login or Join to leave a comment
Login Join