AMSS-NCKU/BSSN_BUILD_CONFIG_MIGRATION.md

BSSN Build Config Migration

This note records the build-configuration fix needed when replacing AMSS_NCKU_Input.py or generate_macrodef.py with a newer upstream version.

Problem

AMSS_NCKU_source/macrodef.h is not the authoritative file used by normal runs. AMSS_NCKU_Program.py first generates macro files under input_data.File_directory, copies AMSS_NCKU_source to <File_directory>/AMSS_NCKU_source_copy, then copies the generated macro files into that copied source tree and compiles there.

Therefore, makefile logic must not depend only on the stale AMSS_NCKU_source/macrodef.h. The actual equation path must be passed to the copied build tree from the same generation step that creates macrodef.h.

The performance regression was caused by compiling/linking the BSSN-EScalar C wrapper into BSSN vacuum builds. For BSSN vacuum (ABEtype=0), the build must use:

BSSN_USE_TRANSFER_CACHE=1
BSSN_USE_ESCALAR_C_KERNEL=0

and must not link bssn_escalar_rhs_c.o.

Required Migration Steps

1. Add an ABE type helper in generate_macrodef.py

Add a helper that maps input_data.Equation_Class to the numeric ABEtype. Use the same mapping as macrodef.h:

def get_abe_type():
    if ( input_data.Equation_Class == "BSSN" ):
        return 0
    elif ( input_data.Equation_Class == "BSSN-EScalar" ):
        return 1
    elif ( input_data.Equation_Class == "BSSN-EM" ):
        return 3
    elif ( input_data.Equation_Class == "Z4C" ):
        return 2
    else:
        raise ValueError("Equation_Class setting error!!!")

Update generate_macrodef_h() to print #define ABEtype {get_abe_type()} instead of duplicating the if/elif mapping.

2. Generate a makefile fragment

In generate_macrodef.py, add:

def generate_build_config():
    file1 = open(os.path.join(input_data.File_directory, "AMSS_NCKU_build.mk"), "w")
    print("# Generated by generate_macrodef.py; do not edit manually.", file=file1)
    print(f"ABE_TYPE := {get_abe_type()}", file=file1)
    file1.close()

This file is the build-time authority for the equation path.

3. Call and copy the generated build config

In AMSS_NCKU_Program.py, after generating macrodef.h and macrodef.fh, call:

generate_macrodef.generate_build_config()
print(" AMSS-NCKU build config AMSS_NCKU_build.mk has been generated. ")

When copying generated files into AMSS_NCKU_source_copy, also copy:

build_config_path = os.path.join(File_directory, "AMSS_NCKU_build.mk")
shutil.copy2(build_config_path, AMSS_NCKU_source_copy)

4. Make the source makefile consume the generated config

At the top of AMSS_NCKU_source/makefile, after include makefile.inc, add:

-include AMSS_NCKU_build.mk

ABE_TYPE ?= $(shell awk '/^[[:space:]]*\#define[[:space:]]+ABEtype/ {print $$3; exit}' macrodef.h 2>/dev/null)

The generated AMSS_NCKU_build.mk is used during normal Python-driven builds. The fallback keeps manual source-tree builds usable.

5. Gate path-specific build options by ABE_TYPE

Use effective build switches:

ifeq ($(USE_TRANSFER_CACHE),auto)
ifeq ($(ABE_TYPE),0)
EFFECTIVE_USE_TRANSFER_CACHE = 1
else
EFFECTIVE_USE_TRANSFER_CACHE = 0
endif
else
EFFECTIVE_USE_TRANSFER_CACHE = $(USE_TRANSFER_CACHE)
endif

ifeq ($(USE_CXX_ESCALAR_KERNEL),1)
ifeq ($(ABE_TYPE),1)
EFFECTIVE_USE_CXX_ESCALAR_KERNEL = 1
else
EFFECTIVE_USE_CXX_ESCALAR_KERNEL = 0
endif
else
EFFECTIVE_USE_CXX_ESCALAR_KERNEL = 0
endif

TRANSFER_CACHE_FLAG = -DBSSN_USE_TRANSFER_CACHE=$(EFFECTIVE_USE_TRANSFER_CACHE)
ESCALAR_KERNEL_FLAG = -DBSSN_USE_ESCALAR_C_KERNEL=$(EFFECTIVE_USE_CXX_ESCALAR_KERNEL)

Only add bssn_escalar_rhs_c.o when the effective EScalar C kernel switch is enabled:

ifeq ($(EFFECTIVE_USE_CXX_ESCALAR_KERNEL),1)
CFILES += bssn_escalar_rhs_c.o
endif

6. Use safe transfer-cache default

In AMSS_NCKU_source/makefile.inc, keep:

USE_TRANSFER_CACHE ?= auto

With the effective switch logic above, this enables cached transfer for BSSN vacuum while keeping non-BSSN paths on the uncached path by default.

Verification Checklist

Run these checks after migrating:

python3 -c "import generate_macrodef; generate_macrodef.generate_build_config()"
cat GW150914/AMSS_NCKU_build.mk

For BSSN, the generated file should contain:

ABE_TYPE := 0

Dry-run the copied or source makefile:

make -n -B INTERP_LB_MODE=off ABE | grep -E 'BSSN_USE_TRANSFER_CACHE|BSSN_USE_ESCALAR_C_KERNEL|bssn_escalar_rhs_c'

Expected BSSN result:

-DBSSN_USE_TRANSFER_CACHE=1 -DBSSN_USE_ESCALAR_C_KERNEL=0

and no bssn_escalar_rhs_c.o in the final link command.

Run the full workflow:

python3 AMSS_NCKU_Program.py

For the 10-step BSSN test, compare coordinate output:

python3 - <<'PY'
from pathlib import Path
old = Path('../GW150914-06457/AMSS_NCKU_output/bssn_BH.dat')
new = Path('GW150914/AMSS_NCKU_output/bssn_BH.dat')

def rows(path):
    out = []
    for line in path.read_text().splitlines():
        if not line.strip() or line.lstrip().startswith('#'):
            continue
        out.append([float(x) for x in line.split()])
    return out

ro, rn = rows(old), rows(new)
n = min(len(ro), len(rn))
max_abs = 0.0
for i in range(n):
    for a, b in zip(ro[i], rn[i]):
        max_abs = max(max_abs, abs(a - b))
print(f"old_rows={len(ro)} new_rows={len(rn)} compared_rows={n}")
print(f"max_abs_diff={max_abs:.17g}")
PY

For the validated migration, the first 10 rows matched exactly:

max_abs_diff=0