lovely-tensors 0.0.8

Lovely Tensors

Lovely Tensors

Install

pip install lovely-tensors

How to use

How often do you find yourself debugging PyTorch code? You dump a tensor to the cell output, and see this:

numbers

tensor([[[-0.3541, -0.3369, -0.4054,  ..., -0.5596, -0.4739,  2.2489],
         [-0.4054, -0.4226, -0.4911,  ..., -0.9192, -0.8507,  2.1633],
         [-0.4739, -0.4739, -0.5424,  ..., -1.0390, -1.0390,  2.1975],
         ...,
         [-0.9020, -0.8335, -0.9363,  ..., -1.4672, -1.2959,  2.2318],
         [-0.8507, -0.7822, -0.9363,  ..., -1.6042, -1.5014,  2.1804],
         [-0.8335, -0.8164, -0.9705,  ..., -1.6555, -1.5528,  2.1119]],

        [[-0.1975, -0.1975, -0.3025,  ..., -0.4776, -0.3725,  2.4111],
         [-0.2500, -0.2325, -0.3375,  ..., -0.7052, -0.6702,  2.3585],
         [-0.3025, -0.2850, -0.3901,  ..., -0.7402, -0.8102,  2.3761],
         ...,
         [-0.4251, -0.2325, -0.3725,  ..., -1.0903, -1.0203,  2.4286],
         [-0.3901, -0.2325, -0.4251,  ..., -1.2304, -1.2304,  2.4111],
         [-0.4076, -0.2850, -0.4776,  ..., -1.2829, -1.2829,  2.3410]],

        [[-0.6715, -0.9853, -0.8807,  ..., -0.9678, -0.6890,  2.3960],
         [-0.7238, -1.0724, -0.9678,  ..., -1.2467, -1.0201,  2.3263],
         [-0.8284, -1.1247, -1.0201,  ..., -1.2641, -1.1596,  2.3786],
         ...,
         [-1.2293, -1.4733, -1.3861,  ..., -1.5081, -1.2641,  2.5180],
         [-1.1944, -1.4559, -1.4210,  ..., -1.6476, -1.4733,  2.4308],
         [-1.2293, -1.5256, -1.5081,  ..., -1.6824, -1.5256,  2.3611]]])

Was it really useful?

What is the shape?
What are the statistics?
Are any of the values nan or inf?
Is it an image of a man holding a tench?

import lovely_tensors as lt

lt.monkey_patch()

__repr__()

# A very short tensor - no min/max
numbers.flatten()[:2]

tensor[2] μ=-0.345 σ=0.012 [-0.354, -0.337]

# A slightly longer one
numbers.flatten()[:6].view(2,3)

tensor[2, 3] n=6 x∈[-0.440, -0.337] μ=-0.388 σ=0.038 [[-0.354, -0.337, -0.405], [-0.440, -0.388, -0.405]]

# Too long to show the values
numbers

tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073

spicy = numbers.flatten()[:12].clone()

spicy[0] *= 10000
spicy[1] /= 10000
spicy[2] = float('inf')
spicy[3] = float('-inf')
spicy[4] = float('nan')

spicy = spicy.reshape((2,6))
spicy

tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!

# A zero tensor
torch.zeros(10, 10)

tensor[10, 10] n=100 all_zeros

spicy.verbose

tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!
[[-3.5405e+03, -3.3693e-05,         inf,        -inf,         nan, -4.0543e-01],
 [-4.2255e-01, -4.9105e-01, -5.0818e-01, -5.5955e-01, -5.4243e-01, -5.0818e-01]]

spicy.plain

[[-3.5405e+03, -3.3693e-05,         inf,        -inf,         nan, -4.0543e-01],
 [-4.2255e-01, -4.9105e-01, -5.0818e-01, -5.5955e-01, -5.4243e-01, -5.0818e-01]]

Going .deeper

numbers.deeper

tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073
  tensor[196, 196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
  tensor[196, 196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
  tensor[196, 196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178

# You can go deeper if you need to
dt = torch.randn(3, 3, 5)
dt.deeper(2)

tensor[3, 3, 5] n=45 x∈[-2.490, 2.484] μ=-0.060 σ=0.884
  tensor[3, 5] n=15 x∈[-2.490, 0.421] μ=-0.770 σ=0.789
    tensor[5] x∈[-2.490, 0.421] μ=-1.063 σ=1.035 [-1.048, -2.490, -1.236, 0.421, -0.961]
    tensor[5] x∈[-0.322, 0.400] μ=-0.097 σ=0.307 [-0.322, -0.271, 0.400, 0.002, -0.295]
    tensor[5] x∈[-1.683, -0.716] μ=-1.151 σ=0.400 [-1.298, -1.683, -1.270, -0.716, -0.789]
  tensor[3, 5] n=15 x∈[-0.740, 2.484] μ=0.333 σ=0.841
    tensor[5] x∈[-0.651, 2.484] μ=0.464 σ=1.196 [-0.651, -0.124, 0.332, 0.278, 2.484]
    tensor[5] x∈[-0.740, 1.300] μ=0.309 σ=0.743 [-0.740, 1.300, 0.583, 0.318, 0.081]
    tensor[5] x∈[-0.512, 1.067] μ=0.227 σ=0.675 [0.794, -0.223, -0.512, 1.067, 0.008]
  tensor[3, 5] n=15 x∈[-0.364, 1.862] μ=0.258 σ=0.561
    tensor[5] x∈[-0.364, 0.676] μ=0.108 σ=0.388 [0.676, 0.020, 0.262, -0.364, -0.053]
    tensor[5] x∈[-0.218, 0.858] μ=0.271 σ=0.471 [0.651, 0.180, -0.117, -0.218, 0.858]
    tensor[5] x∈[-0.053, 1.862] μ=0.395 σ=0.822 [1.862, -0.053, 0.003, 0.096, 0.066]

Now in .rgb colour

The important queston - is it our man?

numbers.rgb

Maaaaybe? Looks like someone normalized him.

in_stats = { "mean": (0.485, 0.456, 0.406),
             "std": (0.229, 0.224, 0.225) }
numbers.rgb(in_stats)

It’s indeed our hero, the Tenchman!

.plt the statistics

(numbers+3).plt

(numbers+3).plt(center="mean", max_s=1000)

(numbers+3).plt(center="range")

Without .monkey_patch

lt.lovely(spicy)

tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!

lt.lovely(spicy, verbose=True)

tensor[2, 6] n=12 x∈[-3.541e+03, -3.369e-05] μ=-393.776 σ=1.180e+03 +inf! -inf! nan!
[[-3.5405e+03, -3.3693e-05,         inf,        -inf,         nan, -4.0543e-01],
 [-4.2255e-01, -4.9105e-01, -5.0818e-01, -5.5955e-01, -5.4243e-01, -5.0818e-01]]

lt.lovely(numbers, depth=1)

tensor[3, 196, 196] n=115248 x∈[-2.118, 2.640] μ=-0.388 σ=1.073
  tensor[196, 196] n=38416 x∈[-2.118, 2.249] μ=-0.324 σ=1.036
  tensor[196, 196] n=38416 x∈[-1.966, 2.429] μ=-0.274 σ=0.973
  tensor[196, 196] n=38416 x∈[-1.804, 2.640] μ=-0.567 σ=1.178

lt.rgb(numbers, in_stats)

lt.plot(numbers, center="mean")