Published on

AI Specialist Course Case Study - Week 2 and 3

Authors
  • avatar
    Name
    Jesús Herman
    Twitter

Introduction

Today I dedicated myself to preparing the development environment for the application. Since I want this to be easily developed on any system and manage dependencies without much complication, I'm going to use Python's virtualenv. Now, the first thing I'm going to do is define the project structure.

Project Structure

/path/to/project
|-- app.py
|-- training_model.py # Here we train the model to process images and store the encodings
|-- requirements.txt
|-- face_encodings.pkl # This file stores the encodings after training the model
|-- credentials.json # Credentials to access Google Sheets
|-- templates/
|   `-- index.html # HTML page for the Flask app interface
|-- images/
|   |-- image1.jpg # Face images to train the model
|   |-- image2.jpg
|   `-- ...- ...

Creating and Activating a Python Virtual Environment

I assume you have Python and Virtualenv installed on your computer; if not, you should do that first. I also note that the commands to activate and deactivate a Python virtual environment vary depending on the operating system you use. You have the different options in this link. I developed this on MacOS. I use the default Python version 3 that comes installed.

# Create the environment

python3 -m venv venv

# Activate it

source venv/bin/activate/

Installing Dependencies

We install the dependencies we defined in the previous article:

pip install flask opencv-python opencv-python-headless dlib face_recognition gspread google-auth google-auth-oauthlib google-auth-httplib2

Here began my ordeal which took up my entire day (and part of the third day of project development). The good thing about the problem was that it happened to me on both MacOS and Windows, with or without WSL. So technically the problem is persistent across different platforms. Curiously, it's a problem that doesn't happen when developing inside a Docker container, but since the plan was always to develop in a local environment first and then deploy the container, I went crazy trying to find a solution.

The Damn CMake from pip is Broken

CMake is a tool for packaging software, among other things. In our case, it is a dependency of dlib (which is necessary to use face_recognition), so when installing dlib, it also installs CMake. When trying to install the package with `, we get the following error (Here for example I tried to install it on Windows).

Collecting dlib
  Using cached dlib-19.24.4.tar.gz (3.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: dlib
  Building wheel for dlib (pyproject.toml) ... error
  error: subprocess-exited-with-error

Building wheel for dlib (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [14 lines of output]
      <string>:210: SyntaxWarning: invalid escape sequence '\('
      <string>:211: SyntaxWarning: invalid escape sequence '\('
      <string>:212: SyntaxWarning: invalid escape sequence '\('
      running bdist_wheel
      running build
      running build_ext
      Traceback (most recent call last):
        File "<frozen runpy>", line 198, in _run_module_as_main
        File "<frozen runpy>", line 88, in _run_code
        File "D:\Dev\app-reconocimiento-facial\venv\Scripts\cmake.exe\__main__.py", line 4, in <module>
      ModuleNotFoundError: No module named 'cmake'

      ERROR: CMake must be installed to build dlib

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for dlib

The first thing I did was check if CMake is installed in the virtual environment:

(venv) PS D:\Dev\app-reconocimiento-facial> python -m pip install cmake
Requirement already satisfied: cmake in d:\Dev\dev\app-reconocimiento-facial\venv\lib\site-packages

From here began the odyssey to try and fix it. As I said, I even deployed a Docker container, where I didn't have these problems. The first thing I read was that CMake had to be installed at the system level, not the virtual environment. That didn't help. Then I tried a thousand tricks, played with specific versions of the packages, did everything, and failed again and again. Finally, I found an entry on Stack Overflow (which unfortunately I couldn't find again) where they indicated that the problem lies in the dlib package from pip, which doesn't work correctly. This led me to a Bug report in the dlib repository where they suggested compiling it ourselves within the virtual environment.

Some of these steps will be different for Windows. Here I detail how I did it on MacOS (I assume it will be the same on Linux).

# Install cmake in the virtual environment
pip install cmake

# Check that CMake is installed
which cmake

# Export the path
export CMAKE=$(which cmake)

# Check that it works
cmake --version

# Inside the environment folder, download dlib and compile it
git clone https://github.com/davisking/dlib.git
cd dlib
mkdir build
cd build
cmake ..
cmake –build .

cd .. # Back to the dlib folder

# We'll install it with the following command
python3 setup.py install

# Check that dlib is working
python3 -c "import dlib; print(dlib.__version__)"

With this, I finally managed to get dlib to work! Honestly, it cost me an arm and a leg!!! 🎉🥳

End of Day Two (and Part of Day Three)

As I mentioned at the beginning, solving this problem cost me a little more than a day. I think I was a bit (quite) a rookie. Although I've done development for years (mainly frontend), some things are still very difficult for me to solve. That, when a package doesn't work, you try to compile it yourself is programmer's manual 101 😅. The thing is, I was able to solve the problem, and along the way, I learned new things that I'm sure will serve me in the future. On to day 3!

Oh! Don't forget to check out the repository: https://github.com/jhmarina/app-reconocimiento-facial

Discuss on Twitter • View on GitHub