Joan: /* Basic classification: Classify images of clothing */

2020-02-07T12:46:41Z

Basic classification: Classify images of clothing

Pàgina nova

__TOC__
=Instal·lació (gener 2020)=
*https://www.tensorflow.org/install/pip
TensorFlow 2 packages are available
*'''tensorflow''' —Latest stable release with CPU and GPU support (Ubuntu and Windows)
*'''ptf-nightly''' —Preview build (unstable). Ubuntu and Windows include GPU support.
'''1.''' Install the Python development environment on your system

<pre>
$ python3 --version
Python 3.6.9

$ pip --version (però compte! aquest és el pip2)
pip 19.3.1 from /home/joan/.local/lib/python2.7/site-packages/pip (python 2.7)

$ pip3 --version
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.6)
$ pip3 install --upgrade pip
Collecting pip
...
Installing collected packages: pip
Successfully installed pip-20.0.2
</pre>

<pre>
$ sudo pip3 install -U virtualenv

$ virtualenv --version
16.7.9
</pre>
'''2.''' Create a virtual environment (recommended)

'''virtualenv''' is a tool to create isolated Python environments

Create a new virtual environment by choosing a Python interpreter and making a ./venv directory to hold it:
<pre>
$ virtualenv --system-site-packages -p python3 ./venv
Already using interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /home/joan/venv/bin/python3
Also creating executable in /home/joan/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.
</pre>
Activate the virtual environment using a shell-specific command:
<pre>
$ source ./venv/bin/activate
</pre>
When virtualenv is active, your shell prompt is prefixed with (venv). El prompt queda:
<pre>
(venv) joan@joanHP:~$
</pre>

Install packages within a virtual environment without affecting the host system setup. Start by upgrading pip:
<pre>
(venv) $ pip install --upgrade pip

(venv) $ pip list # show packages installed within the virtual environment
Package Version
--------------------- ---------
apt-clone 0.2.1
apturl 0.5.2
beautifulsoup4 4.6.0
Brlapi 0.6.6
...
</pre>
And to exit virtualenv later:
<pre>
(venv) $ deactivate # don't exit until you're done using TensorFlow
</pre>

'''3.''' Install the TensorFlow pip package
Choose one of the following TensorFlow packages to install from PyPI:

*tensorflow —Latest stable release with CPU and GPU support (Ubuntu and Windows).
*tf-nightly —Preview build (unstable). Ubuntu and Windows include GPU support.
*tensorflow==1.15 —The final version of TensorFlow 1.x.

Instal·larem el primer paquet.
<pre>
(venv) $ pip install --upgrade tensorflow
</pre>

Verify the install:
<pre>
$ python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
</pre>
En comptes d'instal·lar-ho en l'entorn virtual també ho puc instal·lar en el sistema:
<pre>
$ pip3 install --user --upgrade tensorflow # install in $HOME
</pre>
Però el problema és el mateix:
<pre>
$ python3 -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
2020-01-30 17:08:16.519868: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
</pre>
El TensorFlow el tinc ben instal·lat:
<pre>
$ python3 -m pip freeze | grep tensor
tensorboard==2.1.0
tensorflow==2.1.0
tensorflow-estimator==2.1.0
</pre>
però el problema que tinc és el mateix que es documenta aqui:
*https://github.com/tensorflow/tensorflow/issues/36201

Des de l'entorn virtual:
<pre>
(venv)$ pip install tensorflow-cpu
...
Successfully installed tensorflow-cpu-2.1.0
</pre>
I ara sí que funciona:
<pre>
(venv) $ python -c "import tensorflow as tf;"

(venv) $ python -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

2020-01-30 17:22:09.073469: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2394605000 Hz
2020-01-30 17:22:09.074097: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x48ae480 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-01-30 17:22:09.074166: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
tf.Tensor(369.16412, shape=(), dtype=float32)
</pre>
I ara ho instal·lem directament al sistema, sense l'entorn virtual (no caldria):
<pre>
$ pip3 install tensorflow-cpu
...
Successfully installed tensorflow-cpu-2.1.0
</pre>
<pre>
$ python3 -c "import tensorflow as tf;"

$ python3 -c "import tensorflow as tf;print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
</pre>

=TensorFlow Tutorial For Beginners (Belgium traffic signals) (prova 1)=
'''NOTA''': el problema és que aquest tutorial està fent amb TF1 (tot i que és bastant nou), però ara ja es va pel TF2, i de fet quan he fet la instal·lació m'ha quedat instal·lat el TF2, i m'està donant molts problemes. Conclusió: més val centrar-se i mirar un tutorial i exemples de TF2.

*https://www.datacamp.com/community/tutorials/tensorflow-tutorial
Les dades:
*https://btsd.ethz.ch/shareddata/
Al Github:
*https://github.com/datacamp/datacamp-community-tutorials/blob/master/TensorFlow%20Tutorial%20For%20Beginners/TensorFlow%20Tutorial%20For%20Beginners.ipynb

El projecte està a ''~/projectes/tensorflow_tutorial'', i creem un entorn virtual en aquest directori:
<pre>
$ cd ~/projectes/tensorflow_tutorial
$ virtualenv --system-site-packages -p python3 ./venv
Already using interpreter /usr/bin/python3
...
done.
$ source ./venv/bin/activate
(venv) $ PS1="E$ "
E$
</pre>
<pre>
E$ python --version
Python 3.6.9
</pre>
Dependències per al projecte:
<pre>
E$ python belgium_traffic_signs_v1.py
NameError: name 'skimage' is not defined

$ pip3 install scikit-image
</pre>
<pre>
E$ python belgium_traffic_signs_v1.py
NameError: name 'skimage' is not defined
</pre>
Problema a la línia 28:
<pre>
images.append(skimage.data.imread(f))
</pre>
El problema no és que la llibreria no estigui ben instal·lada

El que tinc instal·lat:
<pre>
$ python3 -m pip freeze | grep tensor
tensorboard==2.1.0
tensorflow==2.1.0
tensorflow-cpu==2.1.0
tensorflow-estimator==2.1.0
tensorflow-gpu==2.1.0
</pre>
=TensorFlow Tutorial For Beginners (Belgium traffic signals) (prova 2)=
python i python3, en aquest entorn virtual, és el mateix:
<pre>
E $ python --version
Python 3.6.9
E $ python3 --version
Python 3.6.9
</pre>
<pre>
E $ pip install tensorflow
...
Successfully installed tensorboard-2.1.0 tensorflow-2.1.0 tensorflow-estimator-2.1.0
</pre>
Per veure què se'ns ha instal·lat i quines versions:
<pre>
E $ python3 -m pip freeze | grep tensor
tensorboard==2.1.0
tensorflow==2.1.0
tensorflow-estimator==2.1.0
</pre>
<pre>
E $ python -c "import tensorflow as tf;"
2020-02-03 13:33:29.725233: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-02-03 13:33:29.725330: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-02-03 13:33:29.725346: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly
</pre>
Hi ha dos problemes. Un és relacionat amb ''libnvinfer.so.6'', i l'altre és un problema amb les targetes gràfiques NVIDIA.

-> hi ha algun problema amb els drivers de la targeta gràfica, em sembla.

Però per fer aquest tutorial necessitor tensorflow-1.15, que no és l'últim (estan havent-hi bastants canvis de tensorflow 1 a 2).

Però si vull instal·lar tensorflow-1.15:
<pre>
E $ pip install tensorflow==1.15
Collecting tensorflow==1.15
...
Uninstalling tensorflow-2.1.0:
Successfully uninstalled tensorflow-2.1.0
...
</pre>
<pre>
E $ python3 -m pip freeze | grep tensor

tensorboard==1.15.0
tensorflow==1.15.0
tensorflow-estimator==1.15.1
</pre>
<pre>
E $ python3 -c "import tensorflow as tf;"
</pre>
i ara no em dóna el missatge emprenyador
<pre>
E $ python prova2.py
WARNING:tensorflow:From prova2.py:12: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
</pre>
funciona, tot i que protesta. Estic treballant amb 1.15. Però en la versió 2 no existeixen les Sessions (protesta tot i que treballo amb la versió 1) Puc treballar amb Sessions.

=GPU support i CUDA enabled cards=
*https://www.tensorflow.org/install/gpu
Software requirements

The following NVIDIA® software must be installed on your system:
*NVIDIA® GPU drivers —CUDA 10.1 requires 418.x or higher.
*CUDA® Toolkit —TensorFlow supports CUDA 10.1 (TensorFlow >= 2.1.0)
*CUPTI ships with the CUDA Toolkit.
*cuDNN SDK (>= 7.6)
*(Optional) TensorRT 6.0 to improve latency and throughput for inference on some models.

Però el portàtil no té una targeta NVIDIA, sinó senzillament una targeta gràfica intel:
<pre>
$ lspci -v
...
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09) (prog-if 00 [VGA controller])
Subsystem: Hewlett-Packard Company 2nd Generation Core Processor Family Integrated Graphics Controller
Flags: bus master, fast devsel, latency 0, IRQ 27

</pre>
Què és '''CUDA'''? En poques paraules, CUDA seria la possibiltiat d'utilitzar la CPU de la gràfica per fer operacions matemàtiques. I és per aixó que CUDA és important.
*https://blogs.nvidia.com/blog/2012/09/10/what-is-cuda-2/
==Aclaració, important==
Si no tinc una targeta gràfica NVIDIA, és inútil instal·lar tensorflow-gpu i CUDA.
<pre>
$ lspci | grep Graph
00:02.0 VGA compatible controller: Intel Corporation 2nd Generation Core Processor Family Integrated Graphics Controller (rev 09)

$ lspci | grep graph
</pre>
Per tant, oblidar-se d'aquest tema.

En el cas de què es disposi una gràfica NVIDIA, la idea és que es fa servir per tal de què els càlculs matemàtics vagin més ràpid, doncs el ML necessita processament de càlcul.

=Tutorials oficials=
*https://www.tensorflow.org/tutorials
<pre>
E $ pip install tensorflow
E $ python3 -m pip freeze | grep tensor
tensorboard==2.1.0
tensorflow==2.1.0
tensorflow-estimator==2.1.0
</pre>
<pre>
from __future__ import absolute_import, division, print_function, unicode_literals

# Install TensorFlow

import tensorflow as tf
</pre>
<pre>
E $ python prova1.py
2020-02-03 14:29:09.335379: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-02-03 14:29:09.335477: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-02-03 14:29:09.335493: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly
</pre>
*https://github.com/tensorflow/tensorflow/issues/34329
=Instal·lar tensorflow amb contenedors Docker=
*https://www.tensorflow.org/install/docker
Docker uses containers to create virtual environments that isolate a TensorFlow installation from the rest of the system. TensorFlow programs are run within this virtual environment that can share resources with its host machine (access directories, use the GPU, connect to the Internet, etc.). The TensorFlow Docker images are tested for each release.

Docker is the easiest way to enable TensorFlow GPU support on Linux since only the NVIDIA® GPU driver is required on the host machine (the NVIDIA® CUDA® Toolkit does not need to be installed).

Aquests són els problemes que he tingut. A veure si amb Docker aconsegueixo tenir un entorn estable per fer proves amb ''tensorflow''.

TensorFlow Docker requirements:
#Install Docker on your local host machine: https://docs.docker.com/install/
#For GPU support on Linux, install NVIDIA Docker support. -> aquesta part '''NO''' l'he de fer, jo no tinc una targeta NVIDIA.
Take note of your Docker version with docker -v. Versions earlier than 19.03 require nvidia-docker2 and the --runtime=nvidia flag. On versions including and after 19.03, you will use the nvidia-container-toolkit package and the --gpus all flag. Both options are documented on the page linked above.

<pre>
$ cat /etc/os-release
NAME="Linux Mint"
VERSION="19.2 (Tina)"
ID=linuxmint
ID_LIKE=ubuntu
PRETTY_NAME="Linux Mint 19.2"
VERSION_ID="19.2"
HOME_URL="https://www.linuxmint.com/"
SUPPORT_URL="https://forums.ubuntu.com/"
BUG_REPORT_URL="http://linuxmint-troubleshooting-guide.readthedocs.io/en/latest/"
PRIVACY_POLICY_URL="https://www.linuxmint.com/"
VERSION_CODENAME=tina
UBUNTU_CODENAME=bionic
</pre>
Tinc Linux Mint (Tina). La versió equivalent de Ubuntu és ''bionic''.
<pre>
$ sudo add-apt-repository 'deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable'
</pre>
<pre>
$ sudo apt-get install docker-ce docker-ce-cli containerd.io
</pre>
<pre>
$ sudo docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:9572f7cdcee8591948c2963463447a53466950b3fc15a247fcad1917ca215a2f
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/

For more examples and ideas, visit:
https://docs.docker.com/get-started/
</pre>

For GPU support on Linux, install NVIDIA Docker support: -> aquesta part jo '''NO''' l'he de fer.
*https://github.com/NVIDIA/nvidia-docker
<pre>
$ distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
joan@joanHP:~/projectes/tensorflow_tutorial$ echo $distribution
linuxmint19.2
</pre>
<pre>
$ curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -

$ curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
# Unsupported distribution!
# Check https://nvidia.github.io/nvidia-docker
</pre>
<pre>
$ sudo apt-get update
</pre>
'''Solució per a Linux Mint''':
<pre>
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu18.04/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
</pre>
<pre>
$ sudo apt-get install -y nvidia-container-toolkit
S'està llegint la llista de paquets… Fet
S'està construint l'arbre de dependències
S'està llegint la informació de l'estat… Fet
E: No s'ha trobat el paquet nvidia-container-toolkit
</pre>
No puc instal·lar el ''NVIDIA Docker support'' perquè estic utilitzant Linux Mint, i no li agrada. (Però bàsicament és perquè jo no tinc una targeta NVIDIA).
<pre>
$ sudo systemctl restart docker
</pre>
==Download a TensorFlow Docker image==
Un cop he instal·lat Docker, ara m'he de descarregar la imatge de tensorflow.

Repositori de les imatges:
*https://hub.docker.com/r/tensorflow/tensorflow/
<pre>
*latest The latest release of TensorFlow CPU binary image. Default.
*nightly Nightly builds of the TensorFlow image. (unstable)
*version Specify the version of the TensorFlow binary image, for example: 2.1.0
*devel Nightly builds of a TensorFlow master development environment. Includes TensorFlow source code.
</pre>
Each base tag has variants that add or change functionality:
<pre>
Tag Variants Description
*tag-gpu The specified tag release with GPU support. (See below)
*tag-py3 The specified tag release with Python 3 support.
*tag-jupyter The specified tag release with Jupyter (includes TensorFlow tutorial notebooks)
</pre>
Descarreguem la ''latest stable release'':
<pre>
$ sudo docker pull tensorflow/tensorflow
[sudo] contrasenya per a joan:
Torneu-ho a provar
[sudo] contrasenya per a joan:
Using default tag: latest
latest: Pulling from tensorflow/tensorflow
2746a4a261c9: Pull complete
4c1d20cdee96: Pull complete
0d3160e1d0de: Pull complete
c8e37668deea: Pull complete
e52cad4ccd83: Pull complete
fac2cc420193: Pull complete
3cae0d3239d1: Pull complete
d383bce69b6c: Pull complete
8a90b3d85a18: Pull complete
4baf29878469: Pull complete
4475017ba949: Pull complete
Digest: sha256:cec133a881c752a82953e88f34a1e9f878b5523779666528c5d2bdfefe3ba6ae
Status: Downloaded newer image for tensorflow/tensorflow:latest
docker.io/tensorflow/tensorflow:latest
</pre>
<pre>
$ sudo docker run -it --rm tensorflow/tensorflow python -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
2020-02-03 15:41:29.474266: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-02-03 15:41:29.474925: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-02-03 15:41:29.475005: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2020-02-03 15:41:32.640011: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2020-02-03 15:41:32.640055: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: UNKNOWN ERROR (303)
2020-02-03 15:41:32.640090: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (89bb3e229959): /proc/driver/nvidia/version does not exist
2020-02-03 15:41:32.753478: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2394560000 Hz
2020-02-03 15:41:32.754356: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55abf7ed7ec0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-02-03 15:41:32.754417: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
tf.Tensor(-1021.8001, shape=(), dtype=float32)
</pre>

Per iniciar el docker de ''tensorflow'':
<pre>
$ sudo docker run -it --rm tensorflow/tensorflow

________ _______________
___ __/__________________________________ ____/__ /________ __
__ / _ _ \_ __ \_ ___/ __ \_ ___/_ /_ __ /_ __ \_ | /| / /
_ / / __/ / / /(__ )/ /_/ / / _ __/ _ / / /_/ /_ |/ |/ /
/_/ \___//_/ /_//____/ \____//_/ /_/ /_/ \____/____/|__/

WARNING: You are running this container as root, which can cause new files in
mounted volumes to be created as the root user on your host machine.

To avoid this, run the container by specifying your user's userid:

$ docker run -u $(id -u):$(id -g) args...

root@67ef3a0eea06:/#
</pre>
El python que invoca aquest ''docker'' és el ''python2'', i no té compatibilitat GPU (els errors que em dóna). Si vull solucionar aquestes coses he de crear un altre docker:
<pre>
$ sudo docker pull tensorflow/tensorflow:latest-gpu-py3
</pre>
Són 800MB. (cal fer-ho amb sudo??)

I ara iniciem el docker:
<pre>
$ sudo docker run -it tensorflow/tensorflow:latest-gpu-py3 bash

$ python --version
Python 3.6.9

$ python -c "import tensorflow as tf;"
2020-02-03 16:02:29.487138: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-02-03 16:02:29.904996: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6

$ python -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"
tf.Tensor(-305.15292, shape=(), dtype=float32)
</pre>
Però el missatge només surt la primera vegada
==Petit tutorial de docker (netejar-ho tot)==
*https://www.digitalocean.com/community/tutorials/how-to-remove-docker-images-containers-and-volumes

per llistar els dockers que tinc:
<pre>
$ sudo docker ps -a
$ sudo docker ps -a -q
</pre>
<pre>
$ sudo docker stop $(docker ps -a -q)
</pre>
per aturar:
<pre>
$ sudo docker stop 49b9bb83e82e
</pre>
I per eliminar aquest docker
<pre>
$ sudo docker rm 49b9bb83e82e
</pre>

locate the ID of the images you want to remove:
<pre>
$ sudo docker images -a
REPOSITORY TAG IMAGE ID CREATED SIZE
tensorflow/tensorflow latest-gpu-py3 e2a4af785bdb 3 weeks ago 4.11GB
tensorflow/tensorflow latest 9bf93bf90865 3 weeks ago 2.47GB
nvidia/cuda latest 9e47e9dfcb9a 2 months ago 2.83GB
hello-world latest fce289e99eb9 13 months ago 1.84kB
</pre>
i per eliminar les imatges que no ens interessen:
<pre>
$ sudo docker rmi IMAGE 9bf93bf90865
</pre>
per eliminar-ho tot:
<pre>
$ sudo docker system prune
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all dangling images
- all dangling build cache
</pre>
Si vull eliminar totes les imatges:
<pre>
$ sudo docker rmi $(sudo docker images -a -q)
</pre>
==Linux docker postinstall==
*https://docs.docker.com/install/linux/linux-postinstall/
===Tema sudo===
The Docker daemon binds to a Unix socket instead of a TCP port. By default that Unix socket is owned by the user root and other users can only access it using sudo. The Docker daemon always runs as the root user.

Per tant, és normal haver d'utilitzar el prefix ''sudo''. Si no ho vull fer, es pot treballar directament com a ''root'', o bé crear el grup ''docker'' com es comenta en el tutorial.
==Configure Docker to start on boot==
Most current Linux distributions (RHEL, CentOS, Fedora, Ubuntu 16.04 and higher) use systemd to manage which services start when the system boots. Ubuntu 14.10 and below use upstart.

A Linux Mint, Debian, Ubuntu, per habilitar o deshabilitar docker en l'inici del sistema:
<pre>
$ sudo systemctl enable docker
$ sudo systemctl disable docker
</pre>
Això és important perquè després de la instal·lació queda habilitat i consumeix recursos, a part de què m'ha donat problemes amb la interfície de xarxa inal·làmbrica.

==TensorFlow sense GPU i CUDA==
<pre>
$ sudo docker pull tensorflow/tensorflow:latest-py3

$ sudo docker run -it tensorflow/tensorflow:latest-py3 bash

root@680ddeeaa9d8:/# python --version
Python 3.6.9
</pre>

<pre>
root@680ddeeaa9d8:/# python -c "import tensorflow as tf"
2020-02-05 08:41:00.647333: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-02-05 08:41:00.647773: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-02-05 08:41:00.647807: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
</pre>
Continua protestant, però de fet no és un error.

'''libnvinfer''' té a verure amb TensorRT (is built on CUDA)
*https://developer.nvidia.com/tensorrt

Una possible solució és deshabilitar els warnings:
<pre>
root@680ddeeaa9d8:/# python -c "import os;os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2';import tensorflow as tf"

python -c "import os;os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2';import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

tf.Tensor(1021.23694, shape=(), dtype=float32)
</pre>
<pre>
0 = all messages are logged (default behavior)
1 = INFO messages are not printed
2 = INFO and WARNING messages are not printed
3 = INFO, WARNING, and ERROR messages are not printed
</pre>
En la consola de python (>>>) aquests warnings només surten la primera vegada.
<pre>
>>> from tensorflow.python.client import device_lib
>>> print(device_lib.list_local_devices())

[name: "/device:CPU:0"
device_type: "CPU"
memory_limit: 268435456
locality {
}
incarnation: 16101204069220344270
, name: "/device:XLA_CPU:0"
device_type: "XLA_CPU"
memory_limit: 17179869184
locality {
}
incarnation: 9804796435611854149
physical_device_desc: "device: XLA_CPU device"
]
</pre>
Efectivament jo no tinc una GPU que es pugui fer servir. Ara només cal que no vagi donant warnings.
=Basic classification: Classify images of clothing=
*https://www.tensorflow.org/tutorials/keras/classification
<pre>
sudo docker run --rm -it \
--workdir=/app \
--volume="$PWD":/app \
--volume="/etc/group:/etc/group:ro" \
--volume="/etc/passwd:/etc/passwd:ro" \
--volume="/etc/shadow:/etc/shadow:ro" \
--volume="/etc/sudoers.d:/etc/sudoers.d:ro" \
tensorflow/tensorflow:latest-py3 bash
</pre>
<pre>
apt-get install joe
joe basic_image_classification_v1.py
</pre>
'''basic_image_classification_v1.py''':
<pre>
from __future__ import absolute_import, division, print_function, unicode_literals

# TensorFlow and tf.keras
import tensorflow as tf
from tensorflow import keras

# Helper libraries
import numpy as np

print(tf.__version__)

fashion_mnist = keras.datasets.fashion_mnist

(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()

class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']

train_images.shape
len(train_labels)
train_labels
test_images.shape
len(test_labels)

train_images = train_images / 255.0
test_images = test_images / 255.0

model = keras.Sequential([
keras.layers.Flatten(input_shape=(28, 28)),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dense(10, activation='softmax')
])

model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])

model.fit(train_images, train_labels, epochs=10)

test_loss, test_acc = model.evaluate(test_images, test_labels, verbose=2)

print('\nTest accuracy:', test_acc)

predictions = model.predict(test_images)
predictions[0]
np.argmax(predictions[0])
test_labels[0]
</pre>
<pre>
# python basic_image_classification_v1.py
</pre>
Quasi ha funcionat fins al final (el meu portàtil no és prou potent i no tinc targeta NVIDIA):
<pre>
...
Epoch 10/10
60000/60000 [==============================] - 9s 153us/sample - loss: 0.2382 - accuracy: 0.9115
2020-02-06 13:17:09.182037: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 62720000 exceeds 10% of system memory.
10000/10000 - 1s - loss: 0.3408 - accuracy: 0.8834

Test accuracy: 0.8834
2020-02-06 13:17:10.347075: W tensorflow/core/framework/cpu_allocator_impl.cc:81] Allocation of 62720000 exceeds 10% of system memory.
</pre>
*https://stackoverflow.com/questions/50304156/tensorflow-allocation-memory-allocation-of-38535168-exceeds-10-of-system-memor/55246936#55246936
<pre>
$ docker stats

CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
0bdbf375e0c7 determined_feistel 0.00% 1.797MiB / 3.801GiB 0.05% 5.66kB / 0B 639kB / 0B 1
</pre>
Mentre corre el script es pot veure com el consum de memòria augmenta. Jo no veig que arriba al 10%, però tant se val, la meva màquina no té les característiques per fer aquests càlculs amb soltura. Sembla ser que es pot configurar el docker per acceptar més memòria.

{{Autor}}, febrer 2020

TensorFlow. Python - Historial de revisió

Joan: /* Basic classification: Classify images of clothing */