.. _cython_compilation:

******************
Cython Compilation
******************

``scaldys-project`` can optionally compile selected Python modules to native
Windows extension modules (``.pyd`` files) using Cython and the MSVC compiler.
This is useful for two purposes:

- **Performance** — tight inner loops or CPU-intensive code can run
  significantly faster as native extensions.
- **Obfuscation** — compiled ``.pyd`` files do not ship as readable Python
  source, making it harder to inspect business logic or algorithms.

When to use Cython compilation
================================

Cython compilation is entirely optional.  If ``compiled_modules`` is empty
(the default), ``scaldys-project`` stages the Python source tree as-is for
the subsequent distribution step.  Enable it only for modules where the
trade-offs are worth the added complexity:

- Modules that are on hot paths and have been profiled as bottlenecks.
- Modules that contain proprietary algorithms you do not wish to ship as
  plain source.

Requirements
============

Install the ``[cython]`` extra::

    uv add --dev "scaldys-project[cython]"

This installs ``Cython`` and ``setuptools``.  The MSVC compiler must also be
present; it is provided by *Visual Studio Build Tools* or a full Visual Studio
installation.  The CPython installer does not include a C compiler.

Configuration
=============

Declare modules to compile in ``scaldys-project.toml``:

.. code-block:: toml

    [cython]
    compiled_modules = [
        "myapp.core.engine",
        "myapp.core.crypto",
    ]
    source_root = "src"          # default; omit if your layout uses "src/"

Each entry is a fully-qualified dotted module path, exactly as you would
write it in an ``import`` statement.

How it works
============

The compilation runs as part of ``scaldys-project build windows`` (and
``build all``), before PyInstaller bundling or wheel assembly.
Internally it proceeds in three phases:

Phase 1 — Source staging
-------------------------

The entire source tree under ``source_root`` is copied to
``build/compiled/``.  This staging directory becomes the working tree for
both Cython and PyInstaller, leaving your original source untouched.

.. code-block:: text

    src/myapp/              →   build/compiled/myapp/
        __init__.py                 __init__.py
        core/                       core/
            engine.py                   engine.py   ← will be compiled
            utils.py                    utils.py    ← kept as-is
            crypto.py                   crypto.py   ← will be compiled

Phase 2 — Cython compilation
-----------------------------

``scaldys-project`` invokes its internal compile runner as a subprocess::

    python -P -m scaldys_project.common.compile_runner \
        build/compiled  myapp.core.engine myapp.core.crypto

The runner uses ``Cython.Build.cythonize`` with ``setuptools`` to:

1. Generate C source files from the ``.py`` files (e.g. ``engine.c``).
2. Compile the C sources with MSVC to produce ``.pyd`` files
   (e.g. ``engine.cpython-313-win_amd64.pyd``).
3. Remove the intermediate ``.c`` files.

Phase 3 — Source clean-up
--------------------------

After compilation, ``scaldys-project`` removes the ``.py`` files in
``build/compiled/`` that correspond to compiled modules.  This ensures
PyInstaller picks up the ``.pyd`` extension rather than the Python source:

.. code-block:: text

    build/compiled/myapp/core/
        engine.cpython-313-win_amd64.pyd   ← compiled extension
        engine.py                          ← REMOVED
        crypto.cpython-313-win_amd64.pyd   ← compiled extension
        crypto.py                          ← REMOVED
        utils.py                           ← kept (not in compiled_modules)

Limitations and caveats
========================

MSVC required
-------------

Cython-generated C extensions on Windows must be compiled with the same
compiler used to build CPython itself — MSVC.  If ``cl.exe`` is not on
``PATH`` (typically via a *Developer Command Prompt* or the Visual Studio
environment), compilation will fail with a compiler-not-found error.

Not all Python is valid Cython
------------------------------

Pure-Python files compile with Cython without any changes in most cases.
However, some dynamic Python patterns (heavy use of ``exec``, ``eval``,
runtime ``__import__``, etc.) may behave differently or fail to compile.

Additionally, **match/case statements (PEP 634) are not currently supported** by
the Cython version used by ``scaldys-project``. If your module uses ``match/case``,
you must replace it with an equivalent ``if-elif-else`` structure before
compilation. ``scaldys-project`` automatically checks for this during the
compliance phase and will abort the build if any ``match/case`` syntax is
detected in modules listed in ``compiled_modules``.

Test compiled modules carefully before shipping.

Module dependencies
-------------------

Only the modules listed in ``compiled_modules`` are compiled.  Modules that
import compiled modules work normally — the Python import system resolves
``.pyd`` files just like ``.py`` files.

Debugging compiled modules
--------------------------

Stack traces from compiled modules show the original Python source line
numbers (Cython embeds them).  However, you cannot set breakpoints directly
in a ``.pyd`` file with standard Python debuggers.  Keep the original source
available during development and compile only for release builds.