China’s J-36 jet engineers warn rogue intelligence models invent war and design data

China’s aircraft engineers have warned that artificial intelligence could feed false data into military intelligence, potentially distorting the design and operation of advanced weapons. The warning came from engineers at China’s AVIC Chengdu Aircraft Research and Design Institute, the organization behind the J-20 and a key developer of the next-generation J-36 fighter program. The concern centers on large language models, which can rapidly analyze vast amounts of information but also generate convincing answers that have no factual basis. For fighter aircraft programs, those mistakes could affect assessments of enemy radars, missile capabilities, and electronic warfare systems. Engineers could then make decisions using intelligence that looks credible but fails basic technical checks. Intelligence drives design Military aircraft development begins with a detailed understanding of the threats an aircraft may face. Engineers need accurate information about enemy radar coverage and operating frequencies. They also study missile engagement zones and the capabilities of electronic warfare systems. The location of early-warning aircraft and ground-based air defenses can further shape an aircraft’s design requirements. Those intelligence findings influence decisions involving stealth, sensors, maneuverability and combat range. A flawed assessment could therefore affect the program before engineers even finalize the aircraft’s requirements. Zhang Xianzhe, an engineer connected to China’s Chengdu aircraft design establishment, warned that large language models can invent highly specific military information. That could include stealth fighter dimensions, payload figures, weapon capabilities, top speeds and combat radius. Radar data presents another concern. A military intelligence system may provide incorrect scan ranges or frequency bands while presenting the information with complete confidence. The output can appear convincing enough to escape immediate scrutiny. When AI gets creative The problem extends beyond incorrect specifications. According to Zhang, a military intelligence model could recommend materials that exceed their actual fatigue limits or suggest maneuvers that violate aerodynamic constraints. It could also develop mission plans that send an aircraft beyond its real operational range. Military intelligence creates another layer of risk. Military AI systems may generate fictional bases, military units, or troop movements that never existed. Analysts using those outputs in simulations could then produce false assessments of an opponent’s actual combat capabilities. That creates a difficult problem for defense organizations increasingly interested in using AI to process intelligence faster. A human analyst may retain final authority, but reviewing every AI-generated recommendation becomes harder as the volume and speed of information increase. Errors reach combat The risks are no longer purely theoretical. A high-casualty military strike earlier this year reportedly followed an AI system’s identification of a target as high priority, alongside outdated intelligence. For the U.S. military, the issue could become increasingly important as AI enters intelligence analysis, weapons development and battlefield planning. Zhang suggested several safeguards to reduce the risk of hallucinations. Defense organizations could ground AI systems in trusted military data and build searchable knowledge bases for verification. Clearer prompts could also reduce ambiguity. He also proposed cross-checking conclusions through AI systems that challenge each other’s outputs.The warning came in a paper published on June 20 in Information Studies: Theory & Application, a journal operated by China’s state-owned defense manufacturer Norinco.Get the latest in engineering, tech, space & science - delivered daily to your inbox.Aamir is a seasoned tech journalist with experience at Exhibit Magazine, Republic World, and PR Newswire. With a deep love for all things tech and science, he has spent years decoding the latest innovations and exploring how they shape industries, lifestyles, and the future of humanity.