Technology

Original: https://mp.weixin.qq.com/s/zk0KxuLzhmMJ4LPYW_OHMA

I have to bury my talent in yesterday

A few days ago, DeepSeek v4.1 was released, pushing the capability level of small models to a whole new height. The speed of AI development has far exceeded everyone's expectations. From that initial version of ChatGPT that could only babble in chat and had a context length of only a few thousand tokens, to OpenAI o1, DeepSeek R1, and Kimi K1.5 Thinking which possess reasoning capabilities, it has only been a mere two years; from reasoning models to agents that can now fluently execute commands in various harness tools and complete complex tasks, it has only been a year and a half. It is hard to imagine what AI will look like if we wait another one, two, or three years, how powerful it will be, and whether it will already possess the ability to self-evolve and deeply penetrate fields such as embodied intelligence.

AI is getting better and better at writing kernels

In just one year, he has gone from a small assistant that can only help me check documents, read code, and find bugs, and become an operator master who can read CUDA, PTX and SASS coding independently, analyzes the pause time of each instruction through professional tools, and then independently optimize the operator. I believe that in the near future, it can also have its own ability to independently design operator scheduling, evaluate the performance of different scheduling schemes, and implement and optimize them.

Of course I am proud of the success of DeepSeek v4.1—after all, its main attention kernel was written by me [1], and its excellence is an affirmation of my kernel. However, the wheel of history rolls forward, and the development of technology is unstoppable. I am very clear that in another half a year or a year, the kernels written by AI will most likely be just as good as the ones I write, or even surpass me. AI can think of 300 tokens per second, knock out a line of command in half a second, and finish writing a piece of code in twenty seconds, but I cannot; AI can continuously improve in model depth, thinking intensity, tool invocation volume (frequency of interaction with the environment), and even parallelism, but I cannot.

Human beings have always shown no hesitation when it comes to destroying themselves. Why did I still choose to do my best to optimize kernels even while knowing clearly that "the better I write kernels, the faster the training and inference speed of our new models will be, the faster the model capabilities will progress, and the sooner I will be replaced"? On the one hand, it is indeed because writing kernels is like playing a video game to me, providing me with immense pleasure. The moment I invent a new technology or see the performance of my kernel go up is no less thrilling than a speedrunner breaking their past record. At the same time, when seeing the performance of my kernel far exceed the official kernels of manufacturers, a great sense of pride wells up in my heart. But beyond that, a more important reason is that even if I were to "slack off" or even deliberately throw a wrench into things to delay model training, other companies' models would still develop as usual and ultimately destroy me anyway. "I certainly hope I won't be revolutionized, but if I must be revolutionized, I hope the one revolutionizing me is myself." When everyone is so obsessed with destroying themselves, I have to join this cruel arms race as well.

What about me then?

By the day when the AI's level of writing kernels is genuinely higher than mine, what will happen to me?

My judgment is: I won't quite "lose my job," but I will have to "change professions." My livelihood can still be preserved, but this might lead to me never having the opportunity again to engage in the work I once loved.

I once made a judgment regarding the changes of the era and my personal situation in the future: because the changes of the era are truly too fast (the AI development mentioned above is a great example), I have completely no way of predicting what will happen in five or ten years, but no matter what, I believe that relying on my own vision, judgment, subjective initiative, and intelligence, I can stay at the poker table of the era and once again stand at the crest of the times. However, this judgment can only guarantee that I won't "lose my job," but cannot guarantee that I won't need to "change professions"—or rather, this judgment encourages me to avoid unemployment by changing professions.

What does changing professions represent then? It represents that I need to abandon the field of kernel design, writing, and optimization that I have plowed deeply into for a long time and am full of passion for, and instead become a "mecha pilot" for Agents. Previously, my interests, what I excelled at, and what the industry needed were basically aligned; now, AI has made what I excel at something it excels at even more, and has shifted the industry's demand from "people who can write high-performance kernels" to "people who can use AI to produce high-performance kernels faster." To adapt to the industry's demand, I am bound to abandon the direction I loved before and turn to an unknown new direction. I believe that I can continue to produce kernels with high quality and high efficiency based on my understanding of engineering, upper-layer model requirements, and underlying hardware, and I know I might love this new direction (or might not), but the feeling of having one's passion taken away does indeed feel rather unpleasant. That quiet joy of sitting at a workstation quietly writing kernels for an entire afternoon might become a swan song this summer. I have to bury my talent in yesterday and become a mecha pilot. There are a few more gears in my hands, but a few less rhythms in my heart.

To give a figurative analogy: You are proficient in the art of knitting sweaters, especially skilled in the knitting of various patterns and the matching of various colors. The sweaters you knit are of solid quality and beautiful patterns, and the rich people from miles around come to ask you to knit sweaters for them, by which you earn a good deal of money. At the same time, you thoroughly enjoy the feeling of sitting by the window, brewing a pot of clear tea, looking at the green mountains, clear streams, cattle, sheep, and cooking smoke outside the window, and quietly knitting sweaters for an entire afternoon. But one day, someone invents a magical machine that only requires providing yarn and patterns to automatically knit sweaters; the quality and texture are no inferior to what you knit by hand, and the speed is much faster than yours. You know very well that your peers can easily reach the level you once had by relying on this machine, so you inevitably have to use it too. You also know that relying on the sweater-knitting skills you accumulated over the past twenty years, even if everyone has machines, your knitting speed and quality will still exceed those of your peers. But that interest of listening to the rain by the window, threading the needle, and spending time slowly was ultimately crushed to pieces by the roar of the machine.

I know this is helpless, but there's no way around it. Livelihoods can be preserved, but past passions will most likely have to be abandoned. I am a person who separates rationality and emotion quite well; I can be very rational when I need to deal with problems using rationality, but sometimes I also display an emotional side. I remember when I moved out of the rental apartment I had lived in for a year, I even cried bitterly, reluctant to part with past memories. Saying goodbye today to that era of handwritten kernels and human-brain optimization is undoubtedly even more cruel.

I don't know if any readers have similar feelings, but I suppose this matter can only be left at that.

What about people then?

While AI continues to make progress, I also express concern about some issues:

• Will students today be much more likely to tend to use AI to complete assignments, especially various labs that lean toward practice? Imagine if there are two choices: one is to bitterly spend eight hours completing a lab, perhaps without even getting a full score; the other is to launch an AI model and, at the cost of a fraction of a yuan and a few minutes of time, directly have the AI write full-score code. Which one will most students choose?
• The above point will lead to a serious deficiency in the engineering skills of a large number of students, including the ability to organize code, build systems, think about potential future demands and deal with them in design beforehand, abstract, and so on. Then, against the backdrop of continuously strengthening AI capabilities, are these "engineering skills" still necessary? Will these engineering skills gradually be discarded by the times like the old skill of "proficiently writing x86 assembly," or will they forever hold value like the ability to "understand the complete set of computer systems from software to systems to hardware"? If it's the latter, then it's dangerous—a person with very poor engineering skills, when paired with AI, can produce heaps of spaghetti code at several times the previous efficiency, thereby burying all kinds of hidden troubles in the system and making this world even more makeshift.
• In future society, will power be more important than technology or IQ?

Perhaps these questions will need to be answered by the era itself.

Conclusion

Along with the development of AI, future society may tend toward two extremes: Communism and Cyberpunk 2077. In the former, productivity is greatly liberated, and people's living standards are significantly improved (I'll just write this much otherwise I'm afraid it won't pass censorship); while in the latter, a few tech companies control most resources, and only a tiny minority of people can use the most advanced AI and various technologies to achieve an effect close to "mechanical ascension," while most people can only use very feeble AI. Class mobility will become increasingly difficult to achieve: you must first have the strongest AI to cross classes, forming a vicious cycle.

Do you suppose that if Anthropic company forever holds the world's most advanced AI in its hands, will future society turn into communism or 2077? Do you suppose?

Therefore, I still believe that the most cutting-edge intelligence should be supplied to everyone in an open and cheap manner. I do not trust that Anthropic or OpenAI can do this, and especially do not hope that Anthropic masters the most advanced artificial intelligence or AGI—to exaggerate slightly, its seriousness is no less than letting Hitler master atomic bomb technology before the Allies. This is also why I chose and persist in staying at DeepSeek: we research powerful, fast, and inclusive artificial intelligence and open-source it, which perhaps can pull the world back a bit from the 2077 side.

May everything in the world of the future be well. May all the beauty be blessed.

[1] "Main Attention" only includes MQA attention with head dim = 512, excluding the indexer used to select the top-k important tokens. That part was written by other (also extremely capable) colleagues (and their AI Agents).