So what's the level of effort to create ChatGPT equivalent products?
Is it something where we'll have 100s of competing AIs, or is it gated to only a few large companies? Not up to date on current training/querying costs.
Can these models feasibly be run locally?
Given the large number of competitors already announced to ChatGPT, I fail to see how the space will be easily defensible or monetizable (despite large value add, competitors can easily undercut eachother)
The problem is performance:
- if you have GPUs with > 330GB VRAM, it'll run fast
- otherwise, you'll run from RAM or NVMe, but very slowly - generating one token every few minutes or so (depending on RAM size / NVMe speed)
The future might be brighter: fp8 already exists and halves the RAM requirements (although it's still very hard to get it running), and there is ongoing research on fp4. Even that would still require 84GB of VRAM to run...
> It is remarkable that such large multi-lingual model is openly available for everybody.
Am I the only one thinking that this remark is a insight into societal failure? The model has been trained on global freely available content, anyone who has published on the Web has contributed.
Yet the wisdom gained from our collective knowledge is assumed to be withheld from us. As the original remark was one of surprise, the authors (and our) assumption is that trained models are expected to be kept from us.
I think it’s similar to how search engines keep their ranking formulas secret, and you can’t run your own off a copy of their index.
Yet we also all contributed to it by publishing (and feeding it, for instance by following googles requirements for micro data). But we don’t own any of it.
Main difference with a search engine is that a search engine ultimately links back to you. So the user, interested in more or want to know where it comes from, ends up on your website.
The same is not true for these AI tools. The output could have been contributed by you, someone else, or everyone, or a combination of those, but it'll never be clear who actually contributed and there will be no credit to anyone besides the author(s) of the models.
How much money do we spend contributing to the training set?
Those insights, comments, articles, code example, etc are free to use because we published those on sites that don't own the content but earn from it. If they owned them, the they would be responsible for hate speech.
So our costs for producing the training set is negligible.
If it fits in system memory, is it still faster on GPU than CPU? Does that involve swapping out one layer at a time? Otherwise I'm very curious how it handles the PCIe latency.
Enough system memory to fit 84GB isn't all that expensive...
Yes, the connection between system memory and the GPU isn’t fast enough to keep the compute units fed with data to process. Generally PCIe latency isn’t as much of a problem as bandwidth.
Honestly even if it were to take a few minutes per response, that's likely sufficient for many use cases. I'd get value out of that if it allowed bypassing a paywall. I'm curious how these models end up being monetized/supported financially, as they sound expensive to run at scale.
The required disk space seems the biggest barrier for local.
I also wonder how open.ai etc provides access to these for free. Reminds me of the adage from when Facebook rose to popularity: "if something is free, 'you' are the product". Perhaps to gather lots more conversational training data for fine tuning.
> As part of our commitment to safe and responsible AI, we review conversations to improve our systems and to ensure the content complies with our policies and safety requirements.
>> Will you use my conversations for training?
> Yes. Your conversations may be reviewed by our AI trainers to improve our systems.
He means multiple GPUs in parallel that have a combined VRAM of that size. So around 4 x NVIDIA A100 80GB, which you can get for around $8.4 / hour in the cloud. or 7 x NVIDIA A6000 or A40 48GB for $5.5 / hour
So not exactly cheap or easy yet for the everyday user, but I believe the models will become smaller and more affordable to run, these are just the "first" big research models focused demonstrating some usefulness after that they can be more focus on the size and speed optimizations. There are multiple methods and lot of research into making them smaller with distilling them, converting to lower precision, pruning the less useful weights, sparsifying. Some achieve around 40% size reduction 60% speed improvement with minimal accuracy loss, others achieve 90% sparsity. So there is hope to run them or similar models on a single but powerful computer.
You'd basically need a rack mount server full of Nvidia H100 cards (80 Vram, they cost $40 thousand us dollars each). So... good luck with that?
On the relatively cheap end Nvidia tesla cards are kinda cheap used, 24 gig ones going for ~$200 with architectures from a few years ago. That's still nearly $3000 worth of cards not counting the rest of the whole computer. This isn't really something you can run out home without having a whole "operation" going on.
Down that far, I start to wonder if trinary circuits might become useful again.
fp4 with 1-3-0 would mean 27 values if the first bit were interpreted as binary. But--and an engineer should check me on this cause to me a transistor is a distant abstraction--I think you could double that to 54 values if you were clever with the sign bit and arithmetic circuitry. Maybe push it to 42 if only some of my intuition is wrong.
"At that time [1955], transistors were not yet available, but it was clear that the machine should not use vacuum tubes. Tubes have a short lifespan, and tube-based machines were idle most of the time because they were always being repaired. A tube machine worked at best for several hours, then it was necessary to look for another malfunction. Yuli Izrailevich Gutenmakher built the LEM-1 machine on ferrite-diode elements. The thought occurred to me that since there are no transistors, then you can try to make a computer on these elements. Sobolev, whom everyone respected very much, arranged for me to go on an internship with Gutenmacher. I studied everything in detail. Since I am a radio engineer by education, I immediately saw that not everything should be done the way they did it. The first thing I noticed is that they use a pair of cores for each bit, one working and one compensating. And an idea came to my mind: what if we make the compensation core do work, as well? Then each cell becomes three-state. Consequently, the number of cores in Setun was seven times less than in LEM-1."
But why? There's nothing special about having 4 storage elements. If you want 54 values then 6 bits are going to be just as effective as 4 trits, and easier to implement in every way.
The models which are small enough to run locally perform so badly it’s not worth bothering.
To run inference on the large models the perform decently you need the equivalent of two or three top end graphics cards.
If you're serious about looking into it now, consider looking at this project that lets you run a bunch of independent machines as a cluster for inference using Bloom:
How badly is bad? What sort of output are we talking?
I am asking as I once had a Markov-chain IRC bot* and while it often struggled to string together a sentence, it was quite hilarious sometimes. Absolutely pointless other than the occasional laugh.
Can it form sentences or are those small models completely unusable for anything?
I'm not thinking OpenAI level uses - sort of compare a Postgres cluster to a SQLite file (not literally, conceptually I guess). Can it be used for single tasks in any way?
Could it figure out how to map search terms to URLs for a knowledge base type thing?
Forgive me if these are silly questions. The extent of my knowledge in this field is asking ChatGPT questions and going "that's so cool" when it answers.
* Your phone's predictive text except it finishes the sentence itself based on a word someone in chat used so that it felt on topic.
In my case it also learned how to form sentences from other people talking in chat, in hindsight it's amazing I never had a Tay issue.
I was recently playing with the GPT-2 and GPT-J models. Results are often non-sensical for any practical purposes, but I think can be used for making something fun - similar to your IRC bot idea.
If you are interested in running these models yourself without having a beefy GPU, you can try my custom inference implementation. It's in pure C/C++ without any 3rd party dependencies, runs straight on the CPU and builds very easily. I think it is relatively well optimised. For example, on a MacBook M1 Pro I can run GPT-2 XL (1.5B params) at 42ms/token and GPT-J / GPT-JT (6B params) at 125ms/token.
Here are a couple of generated examples using GPT-J:
These are examples using zero-shot prompt where the model auto-completes a text given a starting prompt. You can try to make a conversation bot with a few-shot prompt, but it's not great. Probably the model needs some fine-tuning for that to become feasible.
I'll have to have a look into those, there's an audience of about 12 people that would be thrilled to hear "eggbot" is back with a shiny modern brain on, haha.
Oddly enough any processing delay is good in an "AI" chat bot, within reason, makes it feel more natural rather than getting a response ping instantly. Chat version of uncanny valley or something, haha.
Something it also did in Markov form was pick randomly from the longest words in the sentence it had decided to reply to, build the rest of it from that, then run itself "backwards" from the picked word to a sentence starter word it knew.
Thank you for the reply! Looking forward to some tinkering.
.... which is actually not of "almost no value." The value of smaller models is different. For example, I have anonymized data, with fields removed. The smaller models do fine for filling those fields in with plausible values.
The smaller models do okay for zero-shot clustering of data in many cases (e.g. liberal versus conservative text), and if not, with minimal training. For generating statistics or probabilistic information about large numbers of text, they're great.
GPT-3, they're not, but I use them in my day-to-day work quite a bit more than I thought I would. I bought a GPU for one purpose, and I find I spin it up a lot these days.
I /really/ want to be able to use a large-scale language model locally, though. For the types of things I'd like it for, such as helping me draft emails, I don't trust OpenAI with my data.
How much of this is the language vs the vast amount of passably accurate domain knowledge? ChatGPT etc. seem magic because they can answer questions about virtually anything with a high degree of plausibility. It often gets specific facts wrong, but the general contours are correct. Many of us know a lot of trivia/specialist knowledge, but I don't think anyone is as broadly informed as ChatGPT appears to be. It's not clear where the language ends and the encyclopedic knowledge starts, but the latter must be taking up a very large amount of the space in the model.
There have been attempts to separate fact knowledge from language knowledge - for example DeepMind RETRO that uses a search index of 1T tokens. RETRO manages to reach GPT-3 performance on some tasks with a 20x smaller model. I believe smaller model are more useful for extractive and classification tasks than creative text generation.
That's what I meant by 'It's not clear where the language ends and the encyclopedic knowledge starts,' since the model (and perhaps our brains) make little distinction.
But the model seems to be storing an absolutely vast amount of information, beyond the the capability of any individual person to accumulate and recall. This is clearly not a prerequisite for language, even if the information is represented linguistically. Put another way, at age 20 I had read maybe 10-20% of what I've read since, but I was capable of reading comprehension and conversation even though my levels of knowledge and insight were much lower. By 'comprehension' I mean in the sense of being able to read a piece of text and answer questions about it or rewrite it, without necessarily having any priors about the topic; the kind of task we expect to be able to assign to a high school graduate.
I'm wondering what the size of an 'ignorant' language model is, as a precursor to more curated/directed training. While the state of the art is very impressive, it's a bit like taking a feast for a thousand people and rendering it into a giant cube of spam. This strategy seems guaranteed to produce a succession of increasingly capable idiots savant but limits other avenues of exploration.
> at age 20 I had read maybe 10-20% of what I've read since, but I was capable of reading comprehension and conversation...
This is because human intelligence is not just language, but lot of indirect context, "software" inside spinal cord (and other non-cortex parts of brain), and even human body itself.
But as I know, current LLMs working in plain flat structures. At the moment, nobody tried to use even neocortex-like structures, not even considered artificial spinal cord.
All these looks like, to teach table lamp, or something similar smart.
Hmmm, 2-3 high end GPUs implies it's likely not very far off from mainstream. Maybe runnable on the average device within 10-20 years... perhaps even sooner if the model/software can be optimized?
ChatGPT prompt: What's the difference between a component and subcomponent?
A component is a self-contained unit of functionality, often in the form of a class or object, that represents a piece of an overall system or application. A subcomponent, on the other hand, is a smaller, simpler component that is used as part of a larger component. A subcomponent is typically used to break down a larger component into smaller, more manageable pieces, and is often used to create reusable building blocks for larger components. In other words, a subcomponent is a component that is used by another component.
it seems a bit long for achieving just running your own AI models locally, given it seems to be largely a question of vram and that you already _could_ do it today with a handful of graphic cards.
20y ago we had the GeForce Ti4400, current graphics cards now come with 100x the vram and 50-60x the bandwidth.
That's one hell of an assumption. Many of my Russian friends were absolutely certain they'll be alive and well for at least the next 20 years not that long ago.
Let's say the 4090ti is priced around $2500. 16 of them would set you back just $40 grand. That's quite a bit for the average consumer, but people spend that kinda dough on cars/boats/planes etc. It's well within the realm of dedicated hobbyists. For companies it's nothing.
Someone will come along selling pre-builts for this purpose, if that's not already available for their current research/enterprise cards.
No, you can get away with under $20k these days. The models can now distribute adequately well, so you can get away with a swarm of thirty two 12GB 3060 Ti cards. 32*$400 = $12,800. Add in computers to stick them in and interlink, and that's sort of where you land.
When you needed this in one machine, the cost to build a box capable of running OPT was around $50-$150k. An 80GB GPU runs around $10k-$20k, and you need five of them.
So you can get away with a good bit less than an NVidia DGX Station. Unless you're a FAANG SWE of a high level of some kind, it's well outside of hobbyist use, but it's not an unreasonable expense for most business use. The way I think of this expense is it's 10% of the annual cost of a SWE. If it improves productivity by more than 10%, it's worth it for a one-person team. With a 10-person team, it's worth it at a 1% improvement.
yeah, that's not what op asked about. That's one of those smaller models that "suck" as you said so confidently in your first comment. Not even close to gpt3/chatgpt. Next time when I talk bullshit I just post a random py file to make my argument. lmao
One thing to consider, there is considerable asymmetry between the high training costs and the lower (relatively) operating costs.
> Perhaps ChatGPT is run with massive operating losses and its all VC subsidized.
I'm not in the know either, but the current access is providing both marketing awareness, and significant human training/feedback to be used for improvements on future commercial projects. It could certainly be the case that that benefits outweigh costs compared to the classic, dumber "get eyeballs now, monetize later" strategy.
This post [0] is a good primer and they are talking about a 20b model gpt3 has 175b. Their other posts go more into details but yes, it really is a massive operation.
Some hard facts from here [1] talking about BLOOM with 175b parameters..
>> Installing the full 175B version is a challenge though as it requires around 350GB of GPU VRAM, which is not something one can easily afford.
But hey why deploy a model like chatGPT at all when you guys can be confidently incorrect all by yourselves.
So it sounds like this is a question of loading the model into VRAM, and not a question of the cost of a single query. I assume once a model is loaded, many queries can be serviced by that model quickly.
There's nothing incorrect about my assertion. If it were to actually take many GPUs to service one query, then there is no mass scale cost viable consumer product. That's just a clear economic fact. Regardless if a model could be theoretically spun up in a cost inefficient manner.
And even 100s of GB of VRAM is not far off from consumer hardware. Look at how quickly graphics ram has expanded over time. About ~10x in ~10 years for high end cards, at a cursory glance at various Nvidia cards. At the same trajectory we could see a 400GB vram card within the next decade (though lots of assumptions)
You really are like a chatbot... look at the last three node sizes and the density of ram in them. It's not gonna happen as fast as you dream about it especially not with the discounts of the last Gens. The hope is to go to fp4 if you want to run it on consumer hardware and we are still not talking to about 2-3 cards. Why not at least try to Google before hammering down on stupid and uninformed hot takes?
Hidden datasets can be replaced with model predictions collected from a public API. So they can be "exfiltrated" from the trained model. And we already maxed out on the accessible online text and the good quality sources.
What is going to make a difference is running models to generate more text for training, because relying on humans alone doesn't scale. For example we could be using LLMs to do brute force problem solving and then fine-tuning on solutions.
AlphaZero is the shining example of a model trained on its own generated data and surpassing us at our own game. The self generated data approach has potential to reach super human levels of performance.
How about illegal datasets like all the phone calls the NSA has been collecting domestically? Someone is going to train a private ChatGPT with that for queries.
THe RWKV model seems really cool. If you could get transformer-like performance with an RNN, the “hard coded” context length problem might go away. (That said, RNNs famously have infinite context in theory and very short context in reality.)
Is there a primer for what RWKV does differently? According to the Github page it seems the key is multiple channels of state with different decaying rates, giving I assume, a combination of short and long term memory. But isn’t that what LSTMs were supposed to do too?
There's already research that tries to fix this problem with transformers in general, like Transformer-XL [1]. I'm a bit puzzled that I don't see much interest in getting a pre-trained model out that uses this architecture---it seems to give good results.
My understanding is that RNNs aren't worse than Transformers per se, they are just slower to train, and use GPU much more efficiently, i.e. much more stuff could be run in parallel.
Yes it is. They were developed to fix the vanishing gradient problem.
The 1997 paper where they were introduced puts it like this:
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient-based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units.
I can confirm it from what we’re seeing on a video prediction task. Future frames end up blurry. The first frame is sharp, but by frame 3 it’s only crisp when it’s very certain of its prediction. Any kind of rare movement, it goes “I kinda know what it roughly looks like” and smears fingerpaint all over the canvas.
The overall trajectory looks ok, so I’ll be more rigorously investigating whether it’s possible to squeeze more precise context out of it. For example, since the first frame is sharp, you could discard the other future frames and use that first frame as the last history entry (rolling completion window). If “the first frame is always sharp” is true, then it seems reasonable that you can generate N sharp frames with that technique, which might work better than predicting N all at once.
You might also mess with your loss function to force it to "make up its mind" as right now the blurry mess likely minimizes the error from the actual frame (which isn't really want you want).
Exactly! That was the exact thing I was trying to think of a way to do.
Got any ideas? There’s discriminators, but after reading over prior work, it seems like they help, but they weren’t really a groundbreaking / effective solution.
I had two harebrained ideas in mind. One is to add yolo style object detection. The difference between a blurry mess and a recognizable object is the fact that it’s a recognizable object, so minimizing the error wrt yolo might work. (“If there are more recognizable objects in the ground truth image than the generated image, penalize the network”)
The other was to try to make some kind of physics-based prediction of the world — if it knows roughly where a street is, or where a wall is relative to an object, then it’ll likely be less confused when generating objects. That idea is very nascent, but right now I’m attacking it by trying to get our RNN to predict an nbody simulation. (Two or three 2D circles that have a gravitational effect on each other, with bouncing when they collide.) The RNN is surprisingly okay at that, even though it’s only examining pixels, but it gets blurry. I was going to try to get it to spit out actual predictions of position, velocity, acceleration, radius in the hopes that it’ll be able to make a connection between “I know there’s a ball flying along this trajectory, so obviously it should still be there 3 frames from now.”
It seems like the more traditional solution is to add a loss term related to the optical flow of the image (displacement from the previous frame to current), or to do foreground/background segmentation masks and have it focus only on the foreground. Both of those feel like partial solutions though, and it feels like there should be some general way to “force it to make up its mind,” as you say. So if you have any oddball ideas (or professional solutions), I’d love to hear!
Have you checked RSSM approach in DreamerV1,V2,V3,PlaNet? It uses deterministic (GRU hidden state) and discrete stochastic latent states. The deterministic and stochastic (sampled) latent state are used to predict the next state. I think the stochastic state might help with your problem a bit.
The readme does not seem to be geared towards people not familiar with the topic.
My questions:
- Is this on the run on consumer GPU scale, or run on 8 A100 scale or you can’t run it yourself ever scale?
- How does it compare to other language models in quality/abilities?
- What is the training data?
It does say on there they are training it on the Pile training data. And they have this bit comparing inference with GPT2-XL:
RWKV-3 1.5B on A40 (tf32) = always 0.015 sec/token, tested using simple pytorch code (no CUDA), GPU utilization 45%, VRAM 7823M
GPT2-XL 1.3B on A40 (tf32) = 0.032 sec/token (for ctxlen 1000), tested using HF, GPU utilization 45% too (interesting), VRAM 9655M
So it looks about twice as fast for inference while using only about 80% as much VRAM. Obviously at such a small size, just 1.5B, you can run it even on consumer GPUs but you could do that with GPT2 as well. If it remains 80% of VRAM usage when scaled up, we’re still talking 282GB once it’s the size of BLOOM w/ 176B parameters. So yeah still 8x A100 40GB cards I guess. Not going to be the Stable Diffusion of LLMs.
Turns out it does not matter if you have transformer/MLP/lstm or whatever, as long as there are enough parameters and training epochs over large dataset things "just work"
This isn't true - the model architecture matters a lot.
In general RNNs cannot handle long term dependencies (ie, long pieces of text) because the gradient vanishes. It's unclear how this solves this problem although they do reference the "attention free transformer" paper: https://arxiv.org/abs/2105.14103
> Transformers achieve remarkable performance in several tasks but due to their quadratic complexity, with respect to the input's length, they are prohibitively slow for very long sequences. To address this limitation, we express the self-attention as a linear dot-product of kernel feature maps and make use of the associativity property of matrix products to reduce the complexity from (N2) to (N), where N is the sequence length. We show that this formulation permits an iterative implementation that dramatically accelerates autoregressive transformers and reveals their relationship to recurrent neural networks. Our linear transformers achieve similar performance to vanilla transformers and they are up to 4000x faster on autoregressive prediction of very long sequences.
To some degree, because we keep recreating the truly essential components the crude "Turing machine completeness" way. In time as we analyze the resulting models, we may find what patterns emerge and optimize for them. The result will be smaller, faster models that perform like larger slower ones.
Give it at least few examples. ~1B networks are not good in zero-shot. Also, don't expect to get answers for things it was not trained on. the_pile is not programming dataset.
RWKV is important because it's fast, it can be trained in parallel and it gives very good results (compared to other networks trained on the same dataset).
Yeah, when the Web was young, and people told URLs to each other, pronouncing "www", which was almost always the prefix of any web server host name also sounded funny.
The leap in public exposure wasn't so much GPT3 to GPT3.5, it was attaching a clean UI to the model, (with sane defaults) and allowing people to talk to it like a person.
Suddenly it became something 'real' then.
(This is purely talking about the public popularity of GPT)
This is mostly correct. GPT3.5 is better, has a larger context window, etc. But it's a very incremental step above GPT3.
I had wired up GPT3 to a Twilio phone number and made something basically like ChatGPT months before ChatGPT was released -- me and my friends texted it all the time to get information, similar to how people use ChatGPT. The prompt to get decent performance is super simple. Just something like:
The following is a transcript between a human and a helpful AI assistant.
The AI assistant is knowledgeable about most facts of the world and provides concise answers to questions.
Transcript:
{splice in the last 30 messages of the conversation}
The next thing the assistant says is:
Over time I did upgrade the prompt a bit to improve performance for specific kinds of queries, but nothing crazy.
Cost me $10-20/mo to run for the low/moderate use by me and a few friends.
Interestingly, for people who didn't know its limitations / how to break it, it was basically passing the turing test. ChatGPT is inhumanly wordy, whereas GPT3 can actually be much more concise when prompted to do so. If, instead of prompting it that it is an AI assistant, you prompt it that it is a close friend with XYZ personality traits, it does a very good job of carrying on a light SMS conversation.
>If [...] you prompt it that it is a close friend with XYZ personality traits
A couple years ago a friend and I trained GPT-2 on our WhatsApp chat history. GPT-2 was more primitive, but it still managed to capture the gist of our personalities and interests, which was equal parts amusing and embarrassing.
We'd have it generate random chats, or ask it questions to see what simulated versions of ourselves would say.
They also did reinforcement learning on top of a frozen trained model. It is considerably more than just attaching a UI as that would just finish sentences compared to answering questions. https://huggingface.co/blog/rlhf
This is a remarkably good take that just didn't dawn on me until I read your comment. Even if ChatGPT had a lesser quality than the current iteration, the fact that they had a way for anyone to easily interact with it really was a homerun, snd can be for any software, really.
I wonder what kind of engineering problems you’ve talked with ChatGPT. Because its limited understanding of spatial problems, I found it hard to use it for mechanical engineering.
How did you do it?
One of the important parts of ChatGPT over plain GPT-3 is the reinforcement learning from human feedback to ensure alignment, without which it's not quite as good of a product for the public.
It's not just the UI. ChatGPT (which is further finetuned and uses RLHF) definitely produces better output than GPT3, especially without prompt engineering.
No. ChatGPT’s UI is incredibly simple and basically exactly what ever chat bot test repl looks like.
The delta of GPT3 -> ChatGPT is from the expanded context and control the model offers through fine tuning. Eg read the instructgpt paper to see the path on the way to ChatGPT.
Well yes, having no context memory, being slightly worse and requiring either a monster rig to run or paying per prompt made it completely and utterly irrelevant.
Even now that it's improved and free to use its actual practical usability is marginal at best given the rate of blatantly wrong info being spewed with 105% confidence at the moment.
> blatantly wrong info being spewed with 105% confidence
There are some approaches. For example in this paper they say truth has a certain logical consistency that is lacking in hallucinations and deception. So they find this latent direction that indicates truth in a frozen LLM. This actually works better than asking the model to self evaluate by text generation, or training with RLHF.
Btw, I think this is one of the deepest discussions about LLM hallucinations and alignment I ever saw. Worth a watch, even if it is a bit long. Not every day something like this comes long.
It makes you wonder what other abstract concepts current models may have had to learn to get as good as they are. If they're doing a good job of modelling when someone is speaking the truth, then what else have they learnt about us?
How complete of a "world model" can you learn purely in a passive way by consuming whatever online text is available to train on, or maybe by consuming all existent written material were it to be digitized? At some point I'm sure you need to be able to interact with the world to test hypothesis etc, but how far can predictive "intelligence" go without that?
I dunno, I played a lot of AI Dungeon when it was GPT3 powered. It was quite nonsencial sometimes. ChatGPT is much better due to reinforcement learning and optimisation they did with it
Is it something where we'll have 100s of competing AIs, or is it gated to only a few large companies? Not up to date on current training/querying costs.
Can these models feasibly be run locally?
Given the large number of competitors already announced to ChatGPT, I fail to see how the space will be easily defensible or monetizable (despite large value add, competitors can easily undercut eachother)