SDLC

Basic Structure Suggestion

# The most basic structure for a code project should look like:
my-model
├── README.md
├── requirements.txt
├── src                <- Source code for this project
└── tests              <- Test code for this project

Advanced Project Structure

Template based on mkrapp/cookiecutter-reproducible-science github

.
├── AUTHORS.md
├── LICENSE
├── README.md
├── bin                <- Your compiled model code can be stored here (not tracked by git)
├── config             <- Configuration files, e.g., for doxygen or for your model if needed
├── data
│   ├── external       <- Data from third party sources.
│   ├── interim        <- Intermediate data that has been transformed.
│   ├── processed      <- The final, canonical data sets for modeling.
│   └── raw            <- The original, immutable data dump.
├── docs               <- Documentation, e.g., doxygen or scientific papers (not tracked by git)
├── notebooks          <- Ipython or R notebooks
├── reports            <- For a manuscript source, e.g., LaTeX, Markdown, etc., or any project reports
│   └── figures        <- Figures for the manuscript or reports
├── src                <- Source code for this project
│   ├── data           <- scripts and programs to process data
│   ├── external       <- Any external source code, e.g., pull other git projects, or external libraries
│   ├── models         <- Source code for your own model
│   ├── tools          <- Any helper scripts go here
│   └── visualization  <- Scripts for visualisation of your results, e.g., matplotlib, ggplot2 related.
└── tests              <- Test code for this project

Virtual Environments

If application A needs version 1.0 of a particular module but application B needs version 2.0, then the requirements are in conflict and installing either version 1.0 or 2.0 will leave one application unable to run.

The solution for this problem is to create a virtual environment, a self-contained directory tree that contains installation for particular versions of software/packages.

Conda

• Conda is an open source package management system and environment management system that runs on Windows, macOS, and Linux.
• It offers dependency and environment management for any language—Python, R, Ruby, Lua, Scala, Java, JavaScript, C/ C++, Fortran, and more.
• Easy user install via Anaconda.

Coding conventions

If your language or project has a standard policy, use that. For example:

• Python: PEP8
• R: Google’s guide for R, tidyverse style guide
• C++: Google’s style guide
• Julia: Official style guide

Example: Black Code Formatter

$ conda install black
$ black myscript.py

# myscript.py:
x = {"a": 37, "b": 42, "c": 927}
y = "hello " + "world"


class foo(object):
    def f(self):
        return y**2

    def g(self, x: int, y: int = 42) -> int:
        return x - -y


def f(a):
    return 37 + -a[42 - a : y * 3]

IDE

Using an Integrated development environment (IDE) will certainly save you time, but the advantages of using an IDE go beyond that. Below are some IDE advantages

1. Syntax highlighting
2. Text autocompletion
3. Refactoring options
4. Easily Importing libraries
5. Build, compile, or run

Visual Studio Code

To install VS Code follow the instructions here.

VSC Example: automatically using black

Configure VSC to use Black: Code (or File) > Preferences > Settings

• Search for python formatting provider and choose black
• Search for format on save and check the box to enable

Select interpreter: View > Command Palette.. (or Ctrl+Shift+P)

• Search for Python: Select Interpreter
• Choose the correct environment

Now the Black package is going to fix your codes layout every time you save a code file.

Version Control

Piled Higher and Deeper by Jorge Cham

Test-driven development

Example, suppose we need to find the result of a number divided by another number:

Possible tests: a_div_b example

Let’s think in all possible scenarios for this problem and how we could test them.

assert a_div_b(4, 2) == 2

assert a_div_b(8, 7) > 1

assert a_div_b(2, 4) == 0.5

assert a_div_b(7, 8) < 1

assert a_div_b(-4, -2) == 2

assert a_div_b(-4, -2) > 0

Bringing it all together

The Hypotenuse Problem

Calculating the hypotenuse

$$c = \sqrt{a^2 + b^2}$$

General Design

• 1 squared function
• 1 sum function
• 1 square root function
• 1 hypotenuse function that uses the other functions

Workflow

1. Install Git, Anaconda, VScode
2. Create a GitHub repository + Licence + .gitignore + Readme
3. Setup GH Action for testing (Python Application)
4. Clone GH repository in local machine
5. Create project structure (source and test folders)
6. Setup tests (start with test_)
7. Develop code
8. Add docstring (you can use autoDocstring - Python Docstring Generator on VS Code)
9. Lint code and tests
10. Push to github
11. EXTRA: Create Sphinx documentation
12. EXTRA: Setup file and local install
13. EXTRA: GH Release

Extra: Sphinx documentation

• Create docstring for every function
• Install sphinx
• Start the basic structure using: $ sphinx-quickstart docs
• Use the apidoc to get docstrings: $ sphinx-apidoc -o docs .
• Edit files:

 import os
 import sys
 sys.path.insert(0, os.path.abspath('../src'))

.. toctree::
   :maxdepth: 2
   :caption: Contents:

   dependencies
   usage
   functions

Dependencies
============

- python
- pytest
- flake8
- black
- sphinx

Usage Guide
============

To start working with this repository you need to clone it onto your local
machine: ::

    $ git clone https://github.com/...


Next ...

API reference
=============

.. automodule:: calc
   :members:
   :undoc-members:
   :show-inheritance:

Extra: documentation Action

Create a new GH action to create a nice website for your documentation.

• The action is available here
• You may need update GH Actions permissions to allow write
• After a successful documentation action, you need to select gh-pages branch to activate your website