As early as last year "AI psychosis" seemed to refer to people going crazy due to talking to ChatGPT too much. That was a useful term for a real phenomenon! Now it seems like it's been taken over to mean "thinking that AI is promising" which is more of a rhetorical bludgeon and less useful as a concept.
No it's typical. It's just that your stipend is usually just x amount of Dollars + whatever tuition is so you never have to care what it is and you don't pay it directly per se, it's just included in your stipend. Someone pays for it at the end of the day though.
Homelessness and visible homelessness need to be distinguished here. The large majority of homeless people are not the ones you notice on the streets. Most try to be discreet. Some have jobs. A person who lives in their car is considered homeless.
The best measure to reduce homelessness is to provide timely support for people who are being evicted from their homes before they lose their jobs (which they might still have) and before their mental health deteriorates. This is the point at which assistance is most effective. You have heard the saying, "an ounce of prevention is worth a pound of cure". Such programs have been applied to great effect in e.g. London.
The way to respond to people who have experienced chronic homelessness with complications is different and more difficult.
The Simon Community where I live went around the city one December and counted how many rough sleepers there were. I forget the exact figure but it was less than 100. Meanwhile there were thousands classified as homeless due to being in temporary accomodation. And this is a part of the UK well known for having a homelessness problem.
You know oddly enough, if you just put someone up in a real place to live for like a year, that's enough for the majority of people to get back on their feet.
There's a local charity I donate to that does this, they've got I think 10 or 15 apartments they put people up in for 3 months, and offer them various forms of placement assistance while they're there. In the last annual report they had I believe a 95% success rate with people they checked in with a year later.
Unfortunately this is pretty selective evidence, but I know for a fact they don't exclude people on the basis of having mental illness or addiction problems, I've worked with them personally.
That wouldn't be enough to do the job but it would be a great start if it was done right. My point was we've flushed $50B (and likely far far more) and what do we get for it? High gas prices. So hurray for the push for renewables and EVs, but there's nicer ways to do it.
> And you need a bunch of social workers too at minimum
Ok, sure. Remember, we're spending the $50B that's been lit on fire so this gives us more jobs and a happier country. And that money circulates in the economy rather than expatriated profits by the defense contractors.
The "fix everything" button is abolishing zoning laws, and its aggregate cost is negative. Aggregate cost is not the issue preventing problems from being solved.
Given a construction budget of $125,000 per residence for a particularly nice two bedroom single bathroom house of about 1000ft², that's 80,000. Estimates are that there are currently between 750,000 to 800,000 people that have no home right now. Taking the high number of 800,000 that $10,000,000,000 is 10% of those people housed. You could reasonably go down to $30,000 for a build for a single floor house of the same footprint if you used mass produced prefabs, and get 330,000 people housed, or over 41%. Do you realize how much that would uplift things if we suddenly had a 41% reduction in homelessness? Considering that companies like Google and OpenAI are throwing around hundreds of billions of dollars which never get circulated back into the wider economy, spending $10,000,000,000 to bring 330,000 people back into economic and societal participation sounds amazing. That's assuming a somewhat low yearly income of $45,000 per person, adding up to $14,850,000,000 in circulation, or a gain of almost $5,000,000,000 right there. Even if we only achieve half of any of this that's $7,000,000,000 for one year. Two years in and the cost has already been paid back and more.
This comes with the giant caveat that we exclude the external costs of such a huge project, like social welfare visits, probation or monitoring if needed, or even just placement programs. Likely those all combined would be a third of the total cost.
China at least has this. The stuff you get at the Ole Supermarket inside a shopping mall is different from the stuff you get from the little store facing the street on the ground floor of your apartment building.
The price is so low that it makes no real difference at this point. I barely go to my local CostCo (on the edge of being worth the annual fee myself!) because it's so incredibly crowded at all times that the savings are only worth it for more expensive items. In contrast, there are several BJ's nearby and the closest one to me is often blissfully sparse. No idea how they manage to stay in business in that location, but it's really nice.
There is no such thing as a free rider in this context, only a fee rider. After all, other supermarkets do fine without a fee. This is not Amazon Prime with free shipping that we're talking about. Charging for just the item still works.
I think you're vastly underestimating how little of human intent is really encoded in language in a strict sense, and how much nontrivial inference of intents LLMs do every day with simple queries. This used to be an apparently insurmountable barrier in pre-LLM NLP, and now it is just not a problem.
Suppose I'm in a cold room, you're standing next to a heater, and I say "it's cold". Obviously my intent is that I want you to turn on the heater. But the literal semantics is just "the ambient temperature in the room is low" and it has nothing to do with heaters. Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do, often so quickly and effortlessly that we don't notice the complexity of the calculation we did.
Or suppose I say to a bot "tell me how to brew a better cup of coffee". What is encoded in the literal meaning of the language here? Who's to say that "better" means "better tasting" as opposed to "greater quantity per unit input"? Or that by "cup of coffee" I mean the liquid drink, as opposed to a cup full of beans? Or perhaps a cup that is made out of coffee beans? In fact the literal meaning doesn't even make sense, as a "cup" is not something that is brewed, rather it is the coffee that should go into the cup, possibly via an intermediate pot.
If the bot only understands literal language then this kind of query is a complete nonstarter. And yet LLMs can handle these kinds of things easily. If anything they struggle more with understanding language itself than with inferring intent.
> Yet ChatGPT can easily figure out likely intent in situations like this, just as humans do
No, it is not "figuring out" anything, much less like a human might. Every time "I'm cold" appears in the training data, something else occurs after that. ChatGPT is a statistical model of what is most likely to follow "I'm cold" (and the other tokens preceding it) according to the data it has been trained on. It is not inferring anything, it is repeating the most common or one of the most common textual sequences that comes after another given textual sequence.
A slight oversimplification, as LLMs are also capable of generating the most statistically plausible textual sequence, which can be a sequence not found in the dataset but rather a synthesized combination of the likely sequences of multiple preceding sets of tokens, but yes, that is in fact what it is doing. Computer software does what it is programmed to do, and LLMs are not programmed to do logical inference in any capacity but rather operate entirely on probabilities learned from a mind-bogglingly large corpus of text (influenced by things like RLHF, which is still just massaging probabilities).
The claims about solving Erdos problems have been wildly overstated, and notably pushed by people who have a very large financial stake in hyping up LLMs. Nonetheless, I did not say that LLMs are useless. If they are trained on sufficient data, it should not be surprising that correct answers are probabilistically likely to occur. Like any computer software, that makes them a useful tool. It does not make them in any way intelligent, any more than a calculator would be considered intelligent despite being completely superior to human intelligence in accomplishing their given task.
Honestly big noobquestion: isn't math just very very nested patternmatching based on a few foundational operators?
ive always felt, that im bad at math, cause i forget all the rules, but seeing solutions (and knowing the used pattern) always made "sense".
I always thought the hard math problems are so deeply nested or you have to remember trick xyz that people just didnt think about it yet..
The amount of mathematical structures and transformations you can apply (the possible rules) is effectively infinite. Simply remembering the rules might work at first, but you'll soon run into the combinatorial explosion: https://en.wikipedia.org/wiki/Combinatorial_explosion
You could go a step further, and simply say "well, ok, then the LLMs are merely doing some form of incremental/heuristic search!". Yes, but at that point you'd also be hard-pressed to claim that humans themselves are doing anything beyond that. You run out of naturalistic explanations.
If by not up for debate, you mean that it is delusional and literally evidence of psychosis to suggest that computer software is doing something it is not programmed to do, you would be correct. Probabilistic analysis can carry you very, very far in doing something that looks like logical inference at the surface level, but it is nonetheless not logical inference. LLM models have been getting increasingly good at factoring in larger and longer contexts and still managing to generate plausibly correct answers, becoming more and more useful all the while, but are still not capable of logical inference. This is why your genius mathematician AGI consciousness stumbles on trivial logic puzzles it has not seen before like the car wash meme.
> Probabilistic analysis can carry you very, very far in doing something that looks like logical inference at the surface level, but it is nonetheless not logical inference.
A statistical approximation of logical inference (as vague as I state it) could (and will) very well pass for logical inference, at least for the common people, whose logic skills are far from perfect.
Also, humans are certainly not capable of the perfect logical inference you speak of. And I get the irony of what I'm saying with such certitude. Logic is still framed in axioms that are framed in languages, we'll never truly get there. Ah, but absoluteness gets in the way of practicality.
Yet, here we are with a tool, that is maybe not at its prime yet, that equals and beat many human beings at logical inference on some problems that are pragmatically relevant. Should I say symptoms of logical inference at that point?
As to why LLMs capacity for (apparent) logical inference is only limited to specific use cases, I don't have a clue. But I'd like to argue that, humans are like that too.
Well, I'm not clairvoyant, but this is a very easy prediction to make. And we're not talking about decades in the future, this is simply a matter of letting the near-future unfold.
The LLMs are doing this via chat, not by physically standing in a room inferring context. You have to prompt the LLM that you're in a room next to someone saying it's cold, the most likely answer being a desire to have temperature turned up. Of course that won't always be the case. Could be an inside joke, could be a comment with no intent to have the heat adjusted, could be a room where the heat can't be adjusted, could be a reference to someone's personality bringing down the temperature so to speak.
Precisely.. this is what the bozo AI-accelerants don't understand.
What LLM's are is almost like a hacked-means of intuition. Its very impressive no doubt. But ultimately it isn't even close to what the well-trained human can infer at lightning speed when combined with intuition.
The LLM producers really ought to accept their existing investments are ultimately not going to yield the returns necessary for a viable self-sustaining business when accounting for future reinvestment needs, and instead move their focus towards understanding how to marry the human and LLM technology. Anthropic has been better on this front of course. OAI though? Complete diasaster.
> it isn't even close to what the well-trained human can infer at lightning speed when combined with intuition.
It's a lot closer to that than anything was five years ago. Do you really think we're going to be interacting with them the same way five years from now?
This is an empty statement - if you pour in hundreds of billions should you not expect progress?
The question is will these firms be able to continue to spend at that rate. The managers of the firms don’t necessarily have control over that - ultimately punishment in the form of drop in stock price hurts many people involved and will force the management to act in the interest of marginal investors. Even Zuckerberg who has majority control had to concede when meta’s stock cratered to below 100.
I can not simulate my brain, it's a huge stretch to imply this.
But with LLMs - anyone can simulate LLM. LLM can be simulated without any uncertainties in pen and paper and a lot of time. Does it mean that 100 tons of paper plus 100 years of time (numbers are just examples) calculating long formulae makes this pile of paper consiousness? Imho answer is definitive no.
Softmax isn't a loss function. It is used to transform model outputs into positive numbers that sum to 1, so that they can be interpreted as probabilities, and then those numbers are passed into (typically) the cross entropy loss function. I think you mean, which models are trained using some function other than softmax to transform the model outputs. There are a number of alternatives to softmax, such as the ones described here https://www.emergentmind.com/topics/sparsemax
They’re not. Cross entropy loss is E[-log q] where q is a probability. You could convert the model outputs x into probabilities using some other function like q = 1/Z x^2, and compute cross entropy loss just fine.
The mechanics of engines was understood at the beginning of the Industrial Revolution, and they were fully reproducible: all of which is true of LLMs today. An LLM is a bunch of floating point numbers and simple operations on them, all of which are fully known.
But the way that steam engines emergently transformed heat into work was not understood at the beginning of the Industrial Revolution. Figuring this out led to an entire new branch of physics, thermodynamics. Figuring out how big next-token predictors give rise to interesting systems is likely to lead to similarly new ideas.
reply