--- title: Lesson 5 - Server Implementation keywords: fastai sidebar: home_sidebar nb_path: "nbs/course2020/vision/05_Inference_Server.ipynb" ---
{% raw %}
{% endraw %}

Lesson Video:

{% raw %}
{% endraw %} {% raw %}

This article is also a Jupyter Notebook available to be run from the top down. There will be code snippets that you can then run in any environment.

Below are the versions of fastai, fastcore, wwf, and nbdev currently running at the time of writing this:

  • fastai: 2.1.10
  • fastcore: 1.3.13
  • wwf: 0.0.7
  • nbdev: 1.1.5

{% endraw %}

Let's now take what we had before and run inference based on a list of filenames. We'll make a quick script to get the ball rolling for how we want everything to do using nbdev again

We'll want the libraries we've used

{% raw %}
from fastai.vision.all import *
{% endraw %}

Including our new style_transfer.py file

{% raw %}
from wwf.style_transfer import *
{% endraw %}

Let's grab our original style image

{% raw %}
learn = load_learner('myModel', cpu=False)
{% endraw %}

And now we can make and prepare our dataloader with a filename!

{% raw %}
dset = Datasets('cat.jpg', tfms=[PILImage.create])
dl = dset.dataloaders(after_item=[ToTensor()], after_batch=[IntToFloatTensor(), Normalize.from_stats(*imagenet_stats)], bs=1)
{% endraw %} {% raw %}
t_im = dl.one_batch()[0]
{% endraw %}

And get our raw output.

{% raw %}
with torch.no_grad():
  res = learn.model(t_im)
{% endraw %}

Let's wrap this into a function

{% raw %}

get_learner[source]

get_learner(fn, cpu=False)

{% endraw %} {% raw %}
def get_learner(fn, cpu=False):
  return load_learner(fn, cpu=cpu)
{% endraw %} {% raw %}

make_datasets[source]

make_datasets(learn, fns, bs=1)

{% endraw %} {% raw %}
def make_datasets(learn, fns, bs=1):
  cuda = next(learn.model.parameters()).is_cuda
  dset = Datasets(fns, tfms=[PILImage.create])
  if cuda: 
    after_batch = [IntToFloatTensor(), Normalize.from_stats(*imagenet_stats)] 
    dl = dset.dataloaders(after_item=[ToTensor()], after_batch=after_batch, bs=1)
  else: 
    after_batch = [Normalize.from_stats(*imagenet_stats, cuda=False)]
    dl = dset.dataloaders(after_item=[ToTensor()], after_batch=after_batch, bs=1, device='cpu')
  return dl
{% endraw %} {% raw %}
from torchvision.utils import save_image
{% endraw %}

We can write a quick save_im function to save all our outputed tensors to images

{% raw %}

save_im[source]

save_im(imgs:list, path)

Save a ncw*h Tensor into seperate images

{% endraw %} {% raw %}
def save_im(imgs:list, path):
  "Save a n*c*w*h `Tensor` into seperate images"
  [save_image(im, f'{path}/{i}.png') for i, im in enumerate(imgs)]
{% endraw %}

Now let's put it all together

{% raw %}

inference[source]

inference(pkl_name, fnames:list, path:Path, cpu:bool=True)

Grab inference on a model, filenames, and a path to save it to

{% endraw %} {% raw %}
def inference(pkl_name, fnames:list, path:Path, cpu:bool=True):
  "Grab inference on a model, filenames, and a path to save it to"
  path = path/'results'
  path.mkdir(parents=True, exist_ok=True)
  learn = get_learner(pkl_name, cpu)
  if len(fnames) > 1:
    dls = []
    for fname in fnames:
      dls.append(make_datasets(learn, fnames, 1))
  else:
    dls = [make_datasets(learn, fnames, 1)]
  res = []
  for b in dls:
    t_im = b.one_batch()[0]
    with torch.no_grad():
      out = learn.model(t_im)
    res.append(out)
  save_im(res, path)
{% endraw %}

And try it out!

{% raw %}
fnames = ['cat.jpg'] * 5
{% endraw %} {% raw %}
inference('myModel', fnames, path=Path(''))
{% endraw %}

Lastly let's make a .py file again to run it off of

And we're done!