Machine Learning News Hubb
Advertisement Banner
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us
Machine Learning News Hubb
No Result
View All Result
Home Machine Learning

How to VSCode, Docker and Python in Sept 2022 | by Matthew Joseph Taylor | Sep, 2022

admin by admin
September 12, 2022
in Machine Learning


MJT + Stable Diffusion 2022

I am an experienced coder but I am not a Python expert.

Recently I’ve been getting more and more into ML, and so have been interacting more with Python.

I’ll be honest and admit that I’ve avoided Python as it frustrates me.

But the only tool that isn’t complained about, is the tool that is never used. Everything useful has its flaws. 🙂

Like a lot of ‘early’ languages, Python has a dependency management and versioning problem. This IMHO is the heart of most issues in Python, and has plagued the language for as long as I’ve been aware of it.

I have adopted the following tools and patterns, which as of today, works well to isolate environments and keep me productive.

I use the following environment variables for development directly on my loptop hardware:

# pipenv create .venv in local project dir similar to node_modules export PIPENV_VENV_IN_PROJECT=1 
export PIPENV_VERBOSITY=-1

The above will create a .venv directory at the root of the project. This allows VSCode to automagically detect the virtual-environment for things like code completion, terminal CLI execution, etc…

The Pipfile itself appears to be ‘ok’ for capturing dependencies. I’m sure there are frustrations around transitive dependency handling, etc, but it works well enough for what it is. Note that the choice of this tool is the the heart of the ‘battle’ for Python, and so this recommendation may well change in the future and is the one I’m least happy with.

Likely one will need to deploy Python code in a linux-like environment.

I’ve personally found it easier over the years to develop in an environment as close to the ‘production environment’ as possible.

Containerized development is a ‘must have’ for ‘serious Python’ IMHO. The main use case for this language (AI/ML) involves using it as a ‘glue language’. Most of the heavy lifting is going to be done in ‘native library’ code. In 2022 ‘native’ === ‘Some specified variant of Linux’.

Setup for containerized development can be found here: https://code.visualstudio.com/docs/remote/containers-tutorial

NOTE: You will need to (re)install the Python VSCode plugin as a ‘remote development plugin’ https://marketplace.visualstudio.com/items?itemName=ms-python.python after you’ve created your container.

After initial setup of your ‘remote container’ (which is a bit of a misnomer IMHO):

To .devcontainer/devcontainer.json add:

"remoteEnv": {
"PIPENV_VENV_IN_PROJECT":"1",
"PIPENV_VERBOSITY":"-1"
}

I also recommend adding to the same file:

"postCreateCommand": "touch /root/.bashrc"

As most containers will not be setup for root interactive shell

Recommend using the same Dockerfile one will is in production for development (or at least as close as possible) and using the ‘Remote Container from Dockerfile’ option when setting up the VSCode ‘Remote Container’.

FROM ubuntu:22.04 RUN apt-get update RUN DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata RUN apt-get install -y software-properties-common
RUN add-apt-repository -y ppa:deadsnakes/ppa
RUN apt-get install -y python3.9
RUN apt-get install -y python3.9-distutils
RUN apt-get install -y pip
# Fix annoying deprecated warning Bug https://askubuntu.com/a/1407138
RUN pip install --upgrade --user setuptools==58.3.0 RUN pip install pipenv

Python as used for ML is a ‘glue language’ with an unfortunate story for dependency management and language compatibility.

Fortunately there are tools/techniques such as Docker Containers and ‘python virtual environments’ one can adopt to make the most of a messy situation.

Predictions:

  • Pipenv/Pipfile will likely be replaced, but .venv or its equivalent/ancestor is here to stay and wants to live in the project folder.
  • ‘Remote’ container development will increasingly become a recognized standard development technique.
  • As ML fever gradually matures into a distinct discipline, the need for Python will be replaced as the community converges to a recognized ‘assembly language’ that can be run on any corresponding ‘virtual machine’ (probably based on Tensorflow or ONNX once they agree to something like a ‘POSIX’ I/O layer/standard). Much as WASM/WASI removes the need to use the Javascript language for ‘web development’.
  • Python will continue for the immediate future as a much maligned necessity, and will perform a similar function as ‘Bash’ does today: ugly, universal and ‘good enough’.



Source link

Previous Post

How Medidata used Amazon SageMaker asynchronous inference to accelerate ML inference predictions up to 30 times faster

Next Post

PyMySQL: Connecting Python and SQL for Data Science | by Kenneth Leung | Sep, 2022

Next Post

PyMySQL: Connecting Python and SQL for Data Science | by Kenneth Leung | Sep, 2022

5 quick tips on what to remember when working on your AI solution | by Maciej Ociepka | Responsappility Blog | Sep, 2022

Why is Data Represented with Vectors / Matrices? | by Nabib Ahmed | Sep, 2022

Related Post

Artificial Intelligence

Exploring TensorFlow Model Prediction Issues | by Adam Brownell | Feb, 2023

by admin
February 2, 2023
Machine Learning

Different Loss Functions used in Regression | by Iqra Bismi | Feb, 2023

by admin
February 2, 2023
Machine Learning

How to organize bills? – 3 ways to track bills

by admin
February 2, 2023
Artificial Intelligence

How to decide between Amazon Rekognition image and video API for video moderation

by admin
February 2, 2023
Artificial Intelligence

The Future of AI: GPT-3 vs GPT-4: A Comparative Analysis | by Mohd Saqib | Jan, 2023

by admin
February 2, 2023
Deep Learning

6 Ways To Streamline Tech Hiring With A Recruitment Automation Platform

by admin
February 2, 2023

© 2023 Machine Learning News Hubb All rights reserved.

Use of these names, logos, and brands does not imply endorsement unless specified. By using this site, you agree to the Privacy Policy and Terms & Conditions.

Navigate Site

  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us

Newsletter Sign Up.

No Result
View All Result
  • Home
  • Machine Learning
  • Artificial Intelligence
  • Big Data
  • Deep Learning
  • Edge AI
  • Neural Network
  • Contact Us

© 2023 JNews - Premium WordPress news & magazine theme by Jegtheme.