# Python with VirtualEnv

Managing several needed packages within a shared resource environment is difficult. Users may need different package versions, or even packages that conflict with each other. In this example, we will show you how to deal with `virtualenv` for python modules.

### To setup virtual environment

We start by setting up a virtual environment on your home space

```bash
mkdir example4
virtualenv-3.6 --no-download ./example4/env
```

### To use the virtual environment

```bash
cd example4
source ./example4/env/bin/activate.csh #This step depends on your SHELL 
```

Now, you are in a virtual environment. You may see `[env]` at the beginning of SHELL command. The following commands will tell you which python you are using:

```bash
which python
python --version
```

To play with the virtual environment, you can try with `numpy` which comes with the virtual environment. Here is an example of `numpy_example.py`

```python
import numpy as np

a = np.zeros((2,2))   # Create an array of all zeros
print(a)              # Prints "[[ 0.  0.]
                      #          [ 0.  0.]]"

b = np.ones((1,2))    # Create an array of all ones
print(b)              # Prints "[[ 1.  1.]]"

c = np.full((2,2), 7)  # Create a constant array
print(c)               # Prints "[[ 7.  7.]
                       #          [ 7.  7.]]"

d = np.eye(2)         # Create a 2x2 identity matrix
print(d)              # Prints "[[ 1.  0.]
                      #          [ 0.  1.]]"

e = np.random.random((2,2))  # Create an array filled with random values
print(e)                     # Might print "[[ 0.91940167  0.08143941]
                             #               [ 0.68744134  0.87236687]]"
```

To run

```python
python numpy_example.py
```

Note that, if your job cannot run outside the virtual environment if numpy is not available. 

### To install package

Here is an example of how to install `ephem` (python package for performing high-precision astronomy computations):

```bash
pip install --upgrade pip #do this at least for the first time that you use virtual environment
pip install --upgrade setuptools #do this at least for the first time that you use virtual environment
pip install ephem
```

Here is an example to use ephem to calculate the astrometric geocentric right ascension and declination for Mars on specific date.

```python
#ephem_job.py
import ephem
mars = ephem.Mars()
mars.compute('2021/5/19')
print(mars.ra)
print(mars.dec)
```

You then can simply run `python ephem_job.py` in the virtual environment. You may see

```
7:08:08.24
23:55:30.6
```

### To leave from the virtual environment

```python
deactivate
```

If you try to run again the previous example, but outside the virtual environment, you may see

```
Traceback (most recent call last):
  File "ephem_job.py", line 1, in <module>
    import ephem
ImportError: No module named ephembecause there is no ephem install in the shared environment.
```

It is because we do not have `ephem` installed in the shared (central) environment.

### Using VirtualEnv with Slurm

Here is an example (`example4.slurm`) of the Slurm submission script to run python application on worker node using VirtualEnv set in the frontend node (as the example above). To submit, you can simply use the same submission command `sbatch example4.slurm`

```bash
#!/bin/bash
#
#SBATCH --qos=cu_hpc
#SBATCH --partition=cpu
#SBATCH --job-name=example4
#SBATCH --output=example4_log.txt
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:10:00
#SBATCH --cpus-per-task=1
#SBATCH --mem-per-cpu=1G

module purge

#To handle PATHs
export MYCODEDIR=`pwd`
echo "MYCODEDIR = "$MYCODEDIR
echo "TMPDIR = "$TMPDIR

#To run C++ program
cd $TMPDIR
cp $MYCODEDIR/ephem_job.py .
source $MYCODEDIR/example4/env/bin/activate
python ephem_job.py
deactivate
```


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://esciencecu-twiki.sc.chula.ac.th/slurm/slurm-examples/slurm-python-with-virtualenv.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.