Attention
The transformer mechanism that lets each token look at every other token and pull in relevant context via learned relevance weights. The reason 'it' can find what it refers to three sentences back.
The transformer mechanism that lets each token look at every other token and pull in relevant context via learned relevance weights. The reason 'it' can find what it refers to three sentences back.