TENSOR SHAPE QUEST
EN
한국어
粵語
中文
日本語
ČEŠTINA
⚡
0
🔥
0
N=100
N=768
N=1000
N=2000
A shape
B shape
A
B
(100,20,30)
(30,50)
(100,20,50)
(100,20,30)
(100,30,50)
(100,20,50)
(100,20,30)
(1,30,50)
(100,20,50)
(8,100,20,30)
(30,50)
(8,100,20,50)
(100,20,30)
(30,)
(100,20)
(30,)
(100,30,50)
(100,50)
(100,20,30)
(64,30,50)
X
(B,S,D)
Multi-Head
Attention
Q·Kᵀ·V
(B,h,S,d_k)
Add &
Norm
Feed
Forward
D→D_ff→D
(B,S,D_ff)
Y
(B,S,D)
B — batch
S — sequence
D — embedding
h — heads
1
2
4
8
12
16
D_ff — FFN inner dim
Linear
y =
Wx
Affine
y =
Wx
+
b
Y = X @ W
+ b
🥉
🥈
🥇
🏆
test_tensor_shapes.py · pip install torch · python test_tensor_shapes.py