Google Transformers Decoder - Only Patent: How Should We View It in the Era of LLMs?
Google's patent family of "Attention-based decoder-only sequence transduction neural networks" directly captures the core framework of decoder-only transformers, which have become the de facto standard structure for large language models (LLMs) today.
Especially when you view a series of registered patents starting from US 16/759,690—such as US 11,556,786 B2, US 11,886,998 B2, US 12,271,817 B2, US 12,299,572 B2, US 12,299,573 B2, and US 12,354,005 B2—as a single family, it becomes quite clear how much of Google has patented its LLM.
1. Common Technical Philosophy of Decoder-Only Patents
The technical idea that runs through the entire family is surprisingly simple.
Create a combined sequence that connects the input sequence and the already generated output token, and use it as input to the network. In other words, the input at a specific generation point is a single long token sequence consisting of the input sequence + the output sequence generated up to the current moment.
This coupling sequence is passed through a self-attention decoder neural network. The decoder consists only of multiple layers of masked self-attention layers, with no separate encoder.
At each generation point, the network output is interpreted as a score (probability) distribution for possible output tokens, and based on this distribution, the following tokens are selected.
Here, masking is a key element repeatedly emphasized in independent ports. The output of each timestep can only reference the input sequence and the output tokens that have already been generated; future tokens can never be seen. This is exactly where we organize the autogressive-only transformer structure, commonly discussed in practice, into a patent language.
In summary, this patent family structures the scope of rights by creating a single combined sequence of inputs and outputs, placing them in a pure decoder structure, and generating tokens one by one using masked self-attention alone.
2. Relationship with Existing Transformer Patents
The 2017 paper "Attention is All You Need," known to most engineers, proposes an attention-based sequence transformation model based on an encoder-decoder structure, introducing multi-head self-attention on both sides. The corresponding patent is the "Attention-based sequence transduction neural networks" family, which comprehensively claims typical structures including both encoders and decoders.
On the other hand, the "Attention-based decoder-only sequence transduction neural networks" family in question shares the same priority but reorganizes its structure as follows.
One is to completely remove the encoder and leave only the decoder. Another difference is that instead of encoding the input sequence separately, the input and past outputs are combined into a single sequence and fed into the decoder.
As a result, while the existing family protects the entire attention-based encoder-decoder structure, the decoder-only family can be seen as occupying a more recent generation of LLM frameworks that use combined sequences.
In Europe, not only the encoder-decoder family but also the decoder-only family has completed EP registration. Since Korea has also started with the International Application (PCT), it is necessary to view both families on a flat surface when reviewing domestic FTOs.
3. Structure of the US 16/759,690 series
Starting with priority in 2017, Google has split the same specification into multiple continuation formats, securing numerous patents with different focuses. To sum up the whole picture, it's as follows.
[1] US 11,556,786 B2 (Default Decoder-Only Structure)
Claim 1
- A method of generating an output sequence comprising a plurality of output tokens
from an input sequence comprising a plurality of input tokens,
the method comprising, at each of a plurality of generation time steps:
generating a combined sequence for the generation time step
that includes the input sequence followed by the output tokens
that have already been generated as of the generation time step;
processing the combined sequence using a self-attention decoder neural network,
wherein the self-attention decoder neural network comprises
a plurality of neural network layers
that include a plurality of masked self-attention neural network layers,
and wherein the self-attention decoder neural network is configured
to process the combined sequence through the plurality of neural network layers
to generate a time step output
that defines a score distribution over a set of possible output tokens; and
selecting, using the time step output,
an output token from the set of possible output tokens
as the next output token in the output sequence.
The first registered US 11,556,786 B2 is the basic type that broadly encompasses the decoder-only structure itself. Without major restrictions on the types of input and output tokens, the output sequence generation method is charged around three axes: combined sequences, masked self-attention, and token selection.
[2] US 11,886,998 B2 (Training Method)
Claim 1
- A computer-implemented method of training a self-attention decoder neural network
for processing an input sequence comprising a plurality of input tokens
to generate an output sequence comprising a plurality of output tokens,
wherein the self-attention decoder neural network comprises
a plurality of neural network layers
that include a plurality of masked self-attention neural network layers,
the method comprising, for each of a plurality of iterations:
obtaining one or more training examples,
wherein each training example comprises
a respective training input sequence
and a respective training output sequence;
for each respective training example,
processing, using the self-attention decoder neural network
according to current values of parameters of the self-attention decoder neural network,
a combined sequence comprising
(i) tokens in the respective training input sequence and
(ii) tokens in the respective training output sequence
to generate a respective prediction output that comprises a respective time step output for each of the tokens in the respective training output sequence; and
updating the current values of the parameters of the self-attention decoder neural network based on (i) the respective training output sequences and (ii) the respective prediction outputs.
Later, US 11,886,998 B2 separated training methods for the same structure. For each training example, a combined sequence consisting of input and answer output sequences is input into the network, and parameters are updated by obtaining predicted distributions corresponding to each position in the answer output sequence. This is a summary of the teacher force-based learning loop at the independent term dimension. Not only the model structure but also the training loop itself becomes a separate area of rights.
[3] US 12,271,817 B2 (Specialized in Natural Language LLM)
Claim 1
- A method for generating an output sequence comprising a plurality of output tokens
from an input sequence comprising a plurality of input tokens
selected from a vocabulary that includes natural language tokens,
the method comprising, at each of a plurality of generation time steps:
generating a combined sequence for the generation time step
that includes the input sequence followed by the output tokens
that have already been generated as of the generation time step;
processing the combined sequence using a self-attention decoder neural network
that comprises a plurality of masked self-attention neural network layers,
and wherein the self-attention decoder neural network is configured
to process the combined sequence to generate a time step output
that defines a score distribution over a set of possible output tokens,
wherein the masked self-attention neural network layers are masked
such that the time step output depends only on the input sequence
and the output tokens that have already been generated as of the generation time step
and not on any output tokens that are after the last token
that had already been generated in the output sequence; and
determining an output token using the time step output.
US 12,271,817 B2 specifies that vocabulary containing input tokens is a vocabulary containing natural language tokens, adjusting the scope to more directly correspond to natural language LLMs. While maintaining the same technological philosophy, it can be interpreted as a choice to further clarify its applicability to actual chatbot and text generation services.
[4] US 12,299,572 B2 (including Mixture-of-Experts)
Claim 1
- A method for generating an output sequence comprising a plurality of output tokens
from an input sequence comprising a plurality of input tokens,
the method comprising, at each of a plurality of generation time steps:
generating a combined sequence for the generation time step
that includes the input sequence followed by the output tokens
that have already been generated as of the generation time step;
processing the combined sequence using a self-attention decoder neural network
that comprises (i) one or more masked self-attention neural network layers
and (ii) one or more mixture of expert layers
to generate a time step output that defines a score distribution
over a set of possible output tokens,
wherein the masked self-attention neural network layers are masked
such that the time step output depends only on the input sequence
and the output tokens that have already been generated as of the generation time step
and not on any output tokens that are after the last token
that had already been generated in the output sequence; and
determining an output token using the time step output.
US 12,299,572 B2 is a version that explicitly includes a Mixture-of-Experts (MoE) layer inside the decoder, combining MoE scaling techniques widely used in recent large-scale models. If implementing a decoder structure that uses both the self-attention layer and the MoE layer, the potential conflict with this patent should also be separately examined.
[5] US 12,299,573 B2 (Image generation version)
Claim 1
- A method for processing an input sequence comprising a plurality of input tokens,
the method comprising:
at each of a plurality of generation time steps:
generating a combined sequence for the generation time step
that includes the input sequence followed by output tokens
based on time step outputs that have already been generated
as of the generation time step;
processing the combined sequence using a self-attention decoder neural network
that comprises a plurality of masked self-attention neural network layers,
and wherein the self-attention decoder neural network is configured
to process the combined to generate a time step output,
wherein the masked self-attention neural network layers are masked
such that the time step output depends only on the input sequence
and the output tokens that have already been generated as of the generation time step
and not on any output tokens that are after the last token
that had already been generated; and
generating an output image using the time step outputs
generated at the plurality of generation time steps.
US 12,299,573 B2 directly addresses scenarios where the same structure is applied to image generation. Viewing the output sequence not as simple text but as a token sequence for constructing the output image, and placing the step of generating the image using time step outputs obtained at multiple generation points in the conclusion of the independent claim. This is where you encounter touchpoints with modern services such as text-to-image and vision-language models.
[6] US 12,354,005 B2 (System/Platform Billing)
Claim 1
- A system comprising:
a user computer; and
a computer system comprising one or more computers
and one or more storage devices storing instructions
that when executed by the one or more computers
cause the one or more computers to perform operations comprising:
receiving, from the user computer, input data specifying an input sequence
comprising a plurality of input tokens of a natural language;
at each of a plurality of generation time steps:
generating a combined sequence for the generation time step
that includes the input sequence followed by output tokens
that have already been generated as of the generation time step;
processing the combined sequence using a self-attention decoder neural network
that comprises a plurality of masked self-attention neural network layers,
and wherein the self-attention decoder neural network is configured
to process the combined sequence to generate a time step output; and
determining a respective output token using the time step output; and
providing, to the user computer, output data specifying an output sequence
comprising the output tokens determined for the plurality of generation time steps.
US 12,354,005 B2 is a system/platform bill that covers the entire user computer and server system. It typically describes a cloud-based LLM service architecture where a natural language input sequence is received from the user's terminal, the server's decoder-only network operates to generate the output sequence, and then it is delivered back to the user's terminal.
Think of it as a structure surrounded by four layers: structure, training, domain-specific applications, and platform.
4. Domestic and International Application Status and Practical Significance
The Decoder-Only family has been confirmed to have entered major jurisdictions such as Europe, China, and South Korea through PCT international applications. Several EP patents have already been registered in Europe, and some applications are still undergoing examination and opposition procedures until relatively recently. Since response applications have also been announced and reviewed in Korea, it is highly likely that both the leading and lagging parties will frequently cite them in future AI-related disputes.
Korean practitioners should view the encoder-decoder family and decoder-only family as a single baseline, and first compare the domestic client's LLM structure with this patent family at the claim chart level.
5. FTO and Avoidance Design Perspectives for Korean Companies
When reviewing an actual FTO, you will compare structures on several axes.
Whether the input and past output are simply connected as a combined sequence, or processed separately through a separate buffer or memory structure. Is the structure designed to process with only a decoder without an encoder, or is there a separate preprocessing encoder? Whether the decoder layer configuration consists only of masked self-attention and general FFN, or whether it uses MoE or other specialized sub-networks. Whether to input the entire input and output sequence simultaneously into the network in the training loop, or to use sampling for partial sequences or other learning strategies. Whether the service is delivered on-premises or based on cloud APIs, among other factors.
Since most commercial LLMs recently adopt decoder-only architectures, it is not easy to be completely free from the structure alone. Practically, rather than directly avoiding the structure itself, the strategy is to differentiate yourself at the upper layers above it.
For example, external memory and search for context (RAG), RLHF and safety assessment, prompt scheduling, multi-agent orchestration, and inference optimization (caching, sharding, memory management) are still relatively less saturated areas in the patent landscape. Even when using the same decoder-only framework, one direction is to focus on detailed techniques such as position encoding, token compression, routing strategies, and sparsity control as points of invention.
In terms of training methods, it goes beyond traditional cross-entropy learning based on teacher forcing, enabling independent inventions in curriculum learning, active learning, domain-adaptive algorithms, and more. In this case, it is advisable to clearly design a loss configuration and data sampling strategy that distinguishes it from Google's training method patent.
6. Implications for Our Filing Strategy
This patent group goes beyond just well-written patents by Google, covering AI· It provides several important signals to Korean practitioners responsible for LLM-related inventions.
First, the power of a continuation/divisional strategy that divides structure, training, application, and system from a single priority point into multiple patents. In Korea, it is important to prepare the initial specification as broadly and abundantly as possible so that various claim combinations can be designed later when filing divisional applications or asserting overseas priority.
Second, the strategy is to secure not only the model structure itself but also the training process and service delivery system as separate rights scopes. If domestic applications are possible, in addition to the method–device–recording media triangle, it is necessary to design the training method and service system (platform) as separate independent entries.
Third, the value of claims specialized for application domains where real revenue is generated. Versions specifying specific use cases—such as natural language LLMs, image generation, and multimodal models—enable a much more intuitive claim chart during the infringement assessment phase.
7. Conclusion
In summary, Google's Decoder-Only Transformer patent family has secured a significant portion of the basic framework of the LLM era, and is building a multi-layered patent barrier across four layers: structure, training, application, and system.
From the perspective of Korean patent attorneys and lawyers, this patent group will be used in the future as AI· It is close to an inevitable basic map when dealing with LLM-related cases. Treating transformers too easily as public goods during technical discussions is a significant gap from the actual patent landscape.
In actual client cases, it is necessary to specifically understand the client's model structure, training pipeline, and deployment methods, then create a detailed claim chart based on this patent family, and then design FTO, avoidance design, and filing strategies based on that.
Pine IP Firm will continue to track related case law, European oppositions, and U.S. IPR trends to propose more concrete strategies to determine the position domestic AI companies can secure in the global patent landscape.
