Hacker Newsnew | past | comments | ask | show | jobs | submit | huragok's commentslogin

If I had the capital I’d make an household inference appliance.

No peripherals except Ethernet, integrated compute (cpu+gpu+mem) and secondary storage (+mobo, psu). No accoutrements, just the minimum amount of hardware to run a model as a utility.

Even the appliance faceplate would be a display showing stats like an old HiFi stereo.

Edit: something like a series of modules consisting of a RISC-V CPU + Vortex GPGPU + memory


You're describing the mac mini/studio with some facelift.


Yeah but like running linux hopefully


so you have invent unified memory for linux first because that’s the limitation today


Fairly sure most iGPUs these days are zero-copy and can dynamically allocate memory so what does "unified memory" mean to you exactly? A wider bus would be nice but it's not exactly a groundbreaking new invention.


I was actually pretty far off:

> Unified memory in Linux creates a single address space accessible to both the CPU and GPU, eliminating the need to manually copy data between system RAM and video memory. It is enabled via NVIDIA's CUDA, AMD's ROCm/HIP, or generic kernel-level Heterogeneous Memory Management (HMM).

So it does exist and is available for platforms that matter.


It is interesting how apple claimed that "unified memory" is something special, and ppl believed them.

Intel and AMD had been doing this for years already, and had linux support for it from day 1.


Cool. Apple was the only one who managed to ship a consumer device with UMA and RDMA support. 2TB VRAM max over RDMA.


I think the REALLY cool thing about apple's shared memory implementation is the ultra-wide memory bus.

Otherwise, AMD is quite close to what Apple has, and Strix Halo is honestly incredible.

Not sure what RDMA brings to the table.


RDMA increases the inference performance by a significant percentage across devices connected via Thunderbolt 5.4x512 is like a 2TB machine.


Thunderbolt RDMA is slower and higher latency than if apple just gave us PCIe, where we could put a (old) connectX card in for infiniband



"Just". And then GPUs, and RAM? And cooling? Will you really appreciate it when sitting right next to it?


Raspberry Pi and other SBCs, Android phones and practically all of the embedded devices with a display and microprocessor.

All have unified memory. Linux runs just fine on all of those.



Ah ok. I replied to ~45 minutes stale page.


Absolutely, but not under the control of Apple.


Isn't that what what George Hotz is doing over at tiny? https://tinycorp.myshopify.com/


Yes, but for inference. 45k is so far out of the budget of a professional unless you earn ridiculous money and have no dependents.


A professional AI engineer? Earning hundreds of thoundands of dollars a year?


Who lives on one of the coasts where that job likely requires them to be, where rent is $3-5k and mortgages within spitting distance of that, sure.


It could heat your home in the winter and your pool in the summer.


Is warming a pool in the summer real where you live?


Yes. Solar thermal heaters on the roof are common in Florida and other parts of the south. Some people also use heat recovery devices attached to the AC condenser. Further north I've only seen natural gas heating (e.g. in very rich NYC exurbs). The amount of shade over the pool has a big effect.


I think the closest to that in existence is the LLM ASIC designed by Taalas:

https://taalas.com/products/

Unfortunately their chatbot, while amazingly fast, doesn't know anything about the company running it.

Anyway I wouldn't mind an ASIC running a diffusion language model locally. Even if eventually it would become dated. Beats outsourcing all that to a company that's running on VC money which in the future might either perish or worse - dominate the market and charge whatever they wish.


Is that the nvidia spark?


Yes, and a lot of others.

A bit too expensive for a home appliance though, isn't it?


I'm keeping an eye on Tenstorrent for this. Pricing seems like its going to end up being in between a super memory dense unified memory platform, and a purpose built GPU.

Definitely on the edge of what would make sense at home, but its interesting.



I lasted about 25 seconds on that site. Way too much friction for me to endure just trying to figure out what it is


Yeah, I don't know who thought that website was a good idea.


"Login to order"

That's a new one.


I feel like this is some sort of satire? There's no actual information or substance to anything on any page of that site.


the pheriphels support, or the appliance faceplate is tens of dollars, that not where you make the saving

95% of the price is going to be in GPU+CPU+RAM


Sounds like reinventing the home server.


build a Xeon / epyc 4u server. 12 channel ram.


Yes, just a big cool Cerebras wafer for the closet please.


A single wafer comes with 44GB RAM, the reason why Cerebras is so interesting is because the architecture scales up to 1.6PB RAM.


Central heating / thinking.


Capitalism was cool and good before one specific technology was invented.


Everything went off the rails after we invented pottery.


Everything went off the rails after we invented crackpottery.


Everything went off the rails after we invented rails.


We discovered rails, we did not invent them.


Slavery?


Capitalism is what replaced slavery, to generalize. By turning slaves into wage laborer-consumers, capitalists increased market sizes, thereby increasing the absolute size of the profit they get from owning the capital enterprises the laborers operated.

The US civil war is the most explicit demonstration of this process. Northern industrial capitalists ended the southern agrarian slave economy, increasing market size and generating extreme wealth in the latter half of the 19th century.


Eh... The internal combustion engine? Because we had climate change, oligarchs, housing prices, opioids, cryptocurrency, and now on top of all that we have war (edit: with Iran) and LLMs, and all of these can be traced back to extinction-level capitalism.


> ... and now on top of all that we have war ...

There have been war since way before capitalism was invented. Beside war, humans were doing evil things way before capitalism: south america natives torturing kids to extract as many tears as possible from them before killing them, to please the gods, comes to mind, for example.

I'll never understand this fantasy people who hate on capitalism have that the world would be all fine and well if only this one time we did communism [1] right.

[1] or whatever floats your boat


Nostalgia for childhood where you had a community (family) and your needs were taken care of (dependence).


I think it's more than modern companies have, for whatever reason, become extremely scummy. They seem to have no values, ethics, or even regard for their own customers. The only thing that seems to matter to them is squeezing out a few more pennies from people, even if means the likely longer term implosion of their own companies. It's just bizarre to me how many companies, and in different industries, seem to be at war with their own customers.

I hold capitalism with a deep fondness, but I think that's largely because my formative years were in better times. For people who've never known any capitalism except for the mess we currently have? It's quite to see how they can think to themselves that anything must better.


pure Capitalism is law of the jungle

as we will soon find out, when AI will become better at Capitalism than us


Not at all. Capitalism only exists as a product of the state, which acts as a guarantor of individual private property rights. It would not be able to survive without the laws and law enforcement of a sovereign with a monopoly on the use of force.


And unchecked Capitalism coopts the state.


anarchy is also capitalistic in nature


Dumb question: how do I load this onto an ICE40?


I think you'd have to translate it to Verilog then feed it to the existing toolchain.


MbS? MbZ?


the cyc of this current ai winter


Can I bring my own hardware?


Yes!

We'll have to write a guide to explain how to plugin our AI agent, but it can work on any hardware for which the base controls display the right interface.

If you want to chat more about what you need feel free to join our discord!


Just as more than one thing can be bad, you can be against imperialism, stand with oppressed peoples and deeply question the official American story.


Agreed that you don't have to accept the official American story.

You can not be "anti-imperialist" and tolerate this war, however. This is a war of Russian imperialism.

American imperialism is bad, I agree with that. But Russian imperialism is also bad. As was British imperialism.

If you're an anti-imperialism, you must be against the Russian invasion.


I agree?


I wasn't really trying to disagree with you on this. I've seen a sentiment from some "anti-imperialist" voices that end up being primarily anti-western voices, that assume that _only_ the US and the West behave with imperialist policy.

Anyway, I wasn't trying to disagree with you or anything. Just adding my own clarification around the context from some of the other discussions I've had on this topic. See, for example, macanchex's reply to my topic, which places all of the blame for this war on Western powers.


In India, even very right wing people are ambivalent or only somewhat supportive of Putin, whereas the supposedly "anti imperialist" Communists seem to have fallen in love with him. They go to great lengths to localize and translate Russian propaganda.


This might be a case of the enemy of my enemy (capitalist USA) are my friends. Actually they are only the enemy of their enemy and also capitalists, but the hopes of a lifetime can be self delusional.


https://www.thepostil.com/the-military-situation-in-the-ukra...

The dramatic developments we are witnessing today have causes that we knew about but refused to see:

    on the strategic level, the expansion of NATO (which we have not dealt with here);
    on the political level, the Western refusal to implement the Minsk Agreements;
    and operationally, the continuous and repeated attacks on the civilian population of the Donbass over the past years and the dramatic increase in late February 2022.


> on the strategic level, the expansion of NATO (which we have not dealt with here);

This is an Orwellian distortion of language.

Expansion can be interpreted in a literal sense or a metaphorical one. Organizations expand in a metaphorical sense. When we say that a company "expands" by by entering a new market or hiring new talent, we know that this is metaphorical.

NATO is a defensive pact with voluntary membership, but by calling it "expansionist," Russia plays a trick where it evokes the literal sense of expand to transform NATO into the aggressor. It seems fairly plain to me that the real expansionists would be the ones who have literally, physically expanded into a neighboring country by annexing Crimea. The expansionists would be the ones who are currently occupying territory in three foreign nations against the will of those nations' governments.

> and operationally, the continuous and repeated attacks on the civilian population of the Donbass over the past years and the dramatic increase in late February 2022.

By third party estimates, the civillian casualties in the Russian invasion are about two to three orders of magnitude higher than the civilian casualties in Donbass prior to the invasion. This is like slaughtering an entire village because it contains a single murderer. There is absolutely zero ambiguity about where the moral high ground is here.


>By third party estimates, the civillian casualties in the Russian invasion are about two to three orders of magnitude higher than the civilian casualties in Donbass prior to the invasion. This is like slaughtering an entire village because it contains a single murderer. There is absolutely zero ambiguity about where the moral high ground is here.

This does not even include the fact that any civilian casualties in Donbass were either incidents when counterfiring to russian and separatist shelling, or russian provocations.

We can even see this right now in the war: Russia just randomly shells apartment blocks with Grads, while UA only targets real military targets on Russian soil, like Belgorod fuel depots or Taganrog airfield. Russian propaganda doesn't even try to claim that, at least yet.


> the expansion of NATO (which we have not dealt with here)

A voluntary alliance isn't an imperialist agenda (assuming that the sovereign nations have the opportunity to make a truly voluntary decision, without outside coercion).

In fact, many nations have requested voluntarily to join NATO. Ukraine was one such nation, and wasn't allowed in.

Arguing that Russia should get a veto over which defensive alliances other nations join is a pro-imperialist position. You're taking the view that Russia gets to determine the behavior of other nations, because they are "in its security umbrella". I'm sorry, but that position is inherently untenable for an "anti-imperialist".

It's a coherent position for Russian Nationalists, or for believers in Super Power Imperialism.


Wasn't expecting the author to start with their time at Nato. It certainly doesn't match the western narrative, but seems far closer to the Russian narrative (explained cohesively).

He seems to be claiming there wasn't weapons transfer to Donbass, etc when they were initially acting as break away republics. How does he explain MH17? I spent too much time looking at this.


> on the political level, the Western refusal to implement the Minsk Agreements; and operationally, the continuous and repeated attacks on the civilian population of the Donbass over the past years and the dramatic increase in late February 2022.

To think that Ukraine was shelling Donbass and "provoking" after what we seen in the last 50 days is truly a mental distortion.


This is, of course, the Russian take on the war. The Ukrainian version is significantly different. (And given the whole genocide thing, I know which I lean towards. I mean really, NATO, the West, and the Ukrainian government is responsible for Russian war crimes? https://www.reuters.com/resizer/i8u1Zr3pjvon_ZDTHHfs5b6IwlE=...)


Jacques Baud is a former colonel of the General Staff, ex-member of the Swiss strategic intelligence, specialist on Eastern countries. He was trained in the American and British intelligence services. He has served as Policy Chief for United Nations Peace Operations.

On Bucha: https://www.moonofalabama.org/2022/04/if-the-pentagon-can-no...


Truly horrendous link. Your appeal to authority is nonsense, it doesn't take long to find someone with even more commas to dispute it.

> Anyone who is still pushing more weapons into Ukraine or tells Kiev to prolong the war is putting more Ukrainian lives at risk for zero potential gain.

Surely life under Russia will be blissful with no Ukrainians harmed.


Just more victims to add to Communisms 100 trillion others


Small-c communism the theory and large-C Communism the Marxist-Leninist tendency.


Careful now, that sounds like socialism!


Yes, that's the point.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: