DeepSeek (Chinese Al co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget. (2048 GPUs for 2 months, $6M).
For reference, this level of capability is supposed to require clusters of closer to 16K GPUs, the ones being brought up today are more around 100K GPUs. E.g. Llama 3 405B used 30.8M GPU-hours, while DeepSeek-V3 looks to be a stronger model at only 2.8M GPU-hours (~11X less compute). If the model also passes vibe checks (e.g. LLM arena rankings are ongoing, my few quick tests went well so far) it will be a highly impressive display of research and engineering under resource constraints.
Does this mean you don't need large GPU clusters for frontier LLMs? No but you have to ensure that you're not wasteful with what you have, and this looks like a nice demonstration that there's still a lot to get thro with both data and algorithms.
China wants to dominate in Al — and some of its models are already beating their U.S. rivals PUBLISHED MON, DEC 16 2024•6:00 PM EST | UPDATED TUE, DEC 17 2024•9:58 AM EST
Al has become the latest battleground between the U.S. and China, with both sides considering it a strategic technology. Washington continues to restrict China's access to leading-edge chips designed to help power artificial intelligence amid fears that the technology could threaten U.S. national security.
While the focus is on Al models right now, there is also debate over what applications will be built on top of them — and who will dominate this global internet landscape going forward.
"If you assume these frontier base Al models are table stakes, it's about what these models are used for, like accelerating frontier science and engineering technology," Lux Capital's Isford said.
It remains to be seen whether the current threshold strikes the right balance. In November, Tencent released a language model called Hunyuan- Large that outperforms Meta's most powerful variant of Llama 3.1 in several benchmarks. While benchmarks are an imperfect measure for comparing Al models' overall intelligence, Hunyuan- Large's performance is impressive because it was trained using the less powerful, unrestricted Nvidia H20 GPUs, according to research by the Berkeley Risk and Security Lab. "They're clearly getting much better use out of the hardware because of better software," says Ritwik Gupta, the author of the research, who also advises the Department of Defense's Defense Innovation Unit.
Rival Chinese lab's DeepSeek-v3, believed to be the strongest open model available, was also trained using surprisingly little compute. Although there is significant uncertainty about how President-elect Donald Trump will approach Al policy, several experts told TIME in November that they expected export controls to persist-and even be expanded.
Before DeepSeek-v3 was released, the trend had already caught the attention of Eric Schmidt, Google's former CEO and one of the most influential voices on U.S. Al policy. In May 2024, Schmidt had confidently asserted that the U.S. maintained a two-to-three year lead in Al, "which is an eternity in my books!
Yet by November, in a talk at the Harvard Kennedy School, Schmidt had changed his tune. He cited the advances from Alibaba, and Tencent as evidence that China was closing the gap. "This is shocking to me," he said. "I thought the restrictions we placed on chips would keep them back."
How China’s award-winning EUV breakthrough sidesteps US chip ban As US restrictions continue getting tougher, Chinese researchers are coming up with innovative approaches to chip manufacturing.
income sifu, if you ask Deepaeek V3 to identify itself, it will tell you b"I am chat gpt" lad die me, kakakaka. Cut out certain calculations or coverage ken give faster answer lor.
China co like to boast so that ken claim government grant. Go check china EV problems... Self drive Ken lamgar tiang, and balme the driver intervention... Kala
CCP is fooling the world,you see all those China social media, ken oni tok good about China, kenot criticize CCP , Xi Big big and big shots in China,else wil be sensor, invite by police to drink tea, censored and made'socially disappeared ' not going to jail is considered lurky...
May be china GDP of 5% come from these type of export... Iran, Russia., north Korea...
Tok nice want peace dun want war, but sapot those countries and r Terrorist by selling weapon to them, so call win win situation. Let these countries go attack us - Europe . Than china can big other smaller countries. ..
revenge against Brazil revelation of BYD enslaved China workers arr., and at the same time to prepare to buy from us to please Donald Duck. Really need a bully to takes another bully....kakaka
Tksw Sifu China is big buyer of Brazil soybeans maa. Brazil dared to tell Byd enslaved workers. Terima padah. Even pmx doesnt dare to criticise china boh. What say you?
Later other countries buy cheap from Brazil and sell to China. Food is an essential necessity, unless cpp want to starved china ppl (which they deed it before)
SINGAPORE, March 14 (Reuters) - Chinese wheat importers have cancelled or postponed about one million metric tons of Australian wheat imports, trade sources with direct knowledge of the deals said, as growing world stockpiles drag down prices.
The moves come after the U.S. government reported cancellation of more than 500,000 metric tons of U.S. wheat exports last week to China, the world's No. 1 buyer, with international prices trading close three and a half year lows.
A second trader in Singapore said trading companies have vacated shipping slots across several Australian ports, which were booked for cargoes to be shipped to China. Both traders declined to be named because of the sensitivity of the matter.
Refinitiv data shows benchmark Russian wheat export prices slipped below $200 a metric ton ($5.44 per bushel) this week for the first time since August 2020, marking the lowest early March price since 2017.
"Cancelling cargoes is a bearish indicator," Whitelaw said "Whether they are doing it to buy again cheaper or because there's less demand, it is still a bearish view on the market."
About 4% of winter cereals in Russia are in "poor and sparse" condition, the Russian Grain Union said on Monday, compared with a typical rate of 6% at this stage of the season. Near-record Brazilian supplies are weighing on soybean prices.
Exactly right prof. Increasing import tariff is in fact the same as increasing tax and the people inside the country must pay for this "tax" in the prices of the imported goods. China reduces this "tax" for imported goods in order to increase the economy by creating stimulans in reduced consumer prices. It is very clear to anyone with logical mind that the American strategy will not make the usa "great" but on the contrary it will increase the poverty level as people's purchasing power will be diminished. Trade between China and the whole world except with the usa will rise even with the EU countries as the usa is isolating itself from the world. 2025 will turn to be a disaster for the usa and its people.
I'm sure and hope the government in China knows, America is not a normal country and it is not a controlled mechanism that thinks for the common good, This is a country run by a small group who have enormous control power and control over corporations on a scale larger than any European country. And add on top of that all the European countries that are in the pocket of this group that are ready to do anything but not to confront them, the war in Ukraine proved to us how many European countries are the slaves of this force,
What minerals or natural treasures do the Europeans have besides taking from other countries in various ways at zero prices and maintaining a strong economy through technology reserved only for themselves, And to the joy of the world, the Chinese come and build, teach and cooperate whether it's in knowledge or technology and destroy and erase all the advantage the Europeans had over these poor countries, So we have to admit that the Chinese have a difficult task ahead of them and they need very strong cooperation with all the countries that benefit from this change.
US-China trade was a privileged relationship until about 2018. Trump changed that. The US produces less, and China more, particularly in advanced manufactures, in the intervening 9 years. China has been steadily reducing its purchases of US exports (ag, chips, energy) over this period and this will continue. Trump and his advisors (Navarro, Lighthizer et al) can try to expand the policies of his first term but it's likely that they will continue to produce the same results.
Thousands of years ago Chinese gave the world gunpowder, compass, papermaking and printing and they shared it all to the world freely..and western countries used all of them to colonised the world including USA today.. And it's doesn't surprise me to see how fast China catching up after Centuries of Humiliation because throughout China history they are known for their invention and business trade and that's why they only take 30 years to become world's largest economic.. So USA saw China as a threat and used sanctions as weapons, but it's didn't have any effects because they has all the knowledge and capabilities to produce their own products and semiconductor..and most importantly, they have a great relationship with others countries because they only want to do business and they don't have any interest to invade others countries and dominated the world
Chinese are not interested in trade war? As long as war brings benefits, the Chinese like war as much as other peoples do, and the Chinese are very good at war, if not the best at it. The difference between Chinese and Westerners is that due to the influence of thousands of years of Confucianism, the Chinese are more cautious about war and pay more attention to a set of standards when making war decisions. One of the main criteria or principles is not to fight unjust wars. When this standard is reached, the second principle follows, which is not to fight unprepared war.For example, China will not start a war with another nuclear power, such as America, or Russia, because this will never meet the second principle, no matter whether it meets the first criteria. Because any potential nuclear conflict will likely lead to real nuclear conflict, which in turn, leads to total destruction to both sides.
Americans can choose a party but not the policy because all parties work for the oligarchs. The US is a plutocracy, not a democracy. Spending $8 trillion on war but no money for the homeless and the infrastructure is evidence of working for oligarchs, not the people. People in China cannot choose a party. Still, the party would listen to the people and change the policy accordingly because the party is working for the people, not the oligarchs. By the definition of democracy, "by the people, for the people," China is more democratic than the US. So, the US vs. China is plutocracy vs. democracy, and democracy indeed prevails.
It's important to understand that in 1912, China adopted Western democracy when it became a republic (the Republic of China). It did not turn out well. The Chinese learned a very important lesson: Western democracy, pluralistic democracy, will not work for China. Of course, Western democracy did work for Taiwan, but we must also understand that the island of Taiwan is nothing like the massive mainland with its billion plus population and multitude of ethnic minorities. Keeping such a large nation unified is far more difficult than keeping Taiwan unified. At any rate, China has learned never again to entertain the notion of becoming a Western democracy. Not in 1949 when the ROC fell to the Communists, not in 1978 when China opened up to
Edelman Trust Barometer 2023 shows that 89 percent of Chinese trust their government. China has the most trusted government in the world. The Global Happiness 2023 survey from Ipsos shows that China is the happiest country in the world at 91 percent. Latana's Democracy Perception Index 2023 shows that China is one of the most democratic countries in the world, well ahead of USA, Canada, UK, France, Germany, Italy, and Japan. China is #6 at 73 percent.
The only truth staring in the face of US and West now is CN far more advanced and improved in comparison. The decline in CN demographics due to declining births will be mitigated by mass technological adoption. On the other hand, US and the West are stuck with motherhood and misguided ideologies.
America wanted to meet Chinese defence minister Li, but on the other foot, they are continuing to sanction Li. Isn't that hypocricy to disgusting levels? Isn't that trying to stab China at the back while shamelessly smiling in front? America needs to go back to school...at least learn to be more cultured and refined.
Lots of Chinese learn English at school. Lots of them study in US. Lots of American books and movies are translated into Chinese. Yes, Chinese knows American well better than American knows about China.
China learns alot about Singapore (with 70% Chinese majority) and makes the model even better for themselves. Many Chinese government officials and scholars are sent to Singapore each year on learning trips. Imagine a CN with many pockets of better-than-Sg cities. Sg is not a "democracy" in US's terms. But is Sg any lesser than any city in US or West? During Covid, CN locked themselves outwards but remained open inwards with zero covid policy. During the period, the world is fortunate to have CN to be able to continue manufacturing and producing goods for the world. The US and the West gated HuaWei and as a result halted the proliferation of 5G outside CN. But within CN, 5G adoption has leap-frogged with many apps and usage in business and everyday
income Sifu, you copy from where', terbalik cakap one, I guess Russia change food for weapon, and Russia got to cheap sell their produce to get mani...
Laf die me China CCP democratic. Xi big big is kna Own as a king I china...and act like it... So obvious kenot see meh.. but the recently there may be some fight below the water.... Now wait to see whether china ppl ken tahan some more or not. However Chinese ppl has been ruled like that for thousands of years.
Most China ppl has been brainwashed by the government lies, and covered up. They dum know the outside world much, and out side world oso dun know them much , only when it open up more and more 2 sides know each other but because of the government control of free speech, ppl know China less and of course ken know usa better because it is open up. All the media in China is under control and speak onky for government. Boleh land oso have something like that, but maybe not so extreme, stillnken publish if they want to risk their license... But boleh Land now I dunno lar...
Post a Comment
People who like this
New Topic
You should check in on some of those fields below.
Title
Category
Comment
Confirmation
Click Confirm to delete this Forum Thread and all the associated comments.
This book is the result of the author's many years of experience and observation throughout his 26 years in the stockbroking industry. It was written for general public to learn to invest based on facts and not on fantasies or hearsay....
Income
13,171 posts
Posted by Income > 1 week ago | Report Abuse
DeepSeek (Chinese Al co) making it look easy today with an open weights release of a frontier-grade LLM trained on a joke of a budget. (2048 GPUs for 2 months, $6M).