diff --git a/README.md b/README.md index f6c6656f2..a98a7d8fa 100644 --- a/README.md +++ b/README.md @@ -1,31 +1,29 @@ - ```ascii - _ __ _ - (_) / _| (_) - _ _ __ | |_ ___ _ __ ___ _ __ ___ ___ ___ _ __ __ _ _ _ __ ___ - | | '_ \| _/ _ \ '__/ _ \ '_ \ / __/ _ \ __ / _ \ '_ \ / _` | | '_ \ / _ \ - | | | | | || __/ | | __/ | | | (_| __/ |__| | __/ | | | (_| | | | | | __/ - |_|_| |_|_| \___|_| \___|_| |_|\___\___| \___|_| |_|\__, |_|_| |_|\___| - __/ | - |___/ -``` -Inference-Engine -================ +___________.__ __ +\_ _____/|__|____ _/ |_ ______ + | __) | \__ \\ __\/ ___/ + | \ | |/ __ \| | \___ \ + \___ / |__(____ /__| /____ > + \/ \/ \/ +``` +Fiats: Functional inference and training for surrogates +======================================================= +Alternatively, _Fortran inference and training for science_. [Overview](#overview) | [Getting Started](#getting-started) | [Documentation](#documentation) Overview -------- -Inference-Engine supports research in the training and deployment of neural-network surrogate models for computational science. -Inference-Engine also provides a platform for exploring and advancing the native parallel programming features of Fortran 2023 in the context of deep learning. -The language features of interest facilitate loop-level parallelism via the `do concurrent` construct and Single-Program, Multiple Data (SMPD) parallelism via "multi-image" (e.g., multithreaded or multiprocess) execution. -Toward these ends, +Fiats supports research on the training and deployment of neural-network surrogate models for computational science. +Fiats also provides a platform for exploring and advancing the native parallel programming features of Fortran 2023 in the context of deep learning. +The design of Fiats centers around functional programming patterns that facilitate concurrency, including loop-level parallelism via the `do concurrent` construct and Single-Program, Multiple Data (SMPD) parallelism via "multi-image" (e.g., multithreaded or multiprocess) execution. +Towards these ends, -* Most Inference-Engine procedures are `pure` and thus satisfy a language requirement for invocation inside `do concurrent`, -* The network training procedure uses `do concurrent` to expose automatic parallelization opportunities to compilers, and +* Most Fiats procedures are `pure` and thus satisfy a language requirement for invocation inside `do concurrent`, +* The network training procedure use `do concurrent` to expose automatic parallelization opportunities to compilers, and * Exploiting multi-image execution to speedup training is under investigation. -To broaden support for the native parallel features, Inference-Engine's contributors also write compiler tests, bug reports, and patches; develop a parallel runtime library ([Caffeine]); participate in the language standardization process; and provide example inference and training code for exercising and evaluating compilers' automatic parallelization capabilities on processors and accelerators, including Graphics Processing Units (GPUs). +To broaden support for the native parallel features, the Fiats contributors also write compiler tests, bug reports, and patches; develop a parallel runtime library ([Caffeine]); participate in the language standardization process; and provide example inference and training code for exercising and evaluating compilers' automatic parallelization capabilities on processors and accelerators, including Graphics Processing Units (GPUs). Available optimizers: * Stochastic gradient descent and @@ -51,7 +49,7 @@ Getting Started The [example] subdirectory contains demonstrations of several relatively simple use cases. We recommend reviewing the examples to see how to handle basic tasks such as configuring a network training run or reading a neural network and using it to perform inference. -The [demo] subdirectory contains demonstration applications that depend on Inference-Engine but build separately due to requiring additional prerequisites such as NetCDF and HDF5. +The [demo] subdirectory contains demonstration applications that depend on Fiats but build separately due to requiring additional prerequisites such as [NetCDF] and [HDF5]. The demonstration applications - Train a cloud microphysics model surrogate for the Intermediate Complexity Atmospheric Research ([ICAR]) package, - Perform inference using a pretrained model for aerosol dynamics in the Energy Exascale Earth System ([E3SM]) package, and @@ -61,26 +59,21 @@ The demonstration applications Because this repository supports programming language research, the code exercises new language features in novel ways. We recommend using any compiler's latest release or even building open-source compilers from source. The [handy-dandy] repository contains scripts capturing steps for building the [LLVM] compiler suite. -The remainder of this section contains commands for building Inference-Engine with a recent Fortran compiler and the Fortran Package Manager ([`fpm`]) in your `PATH`. - -#### GNU (`gfortran`) 13 or higher required -``` -fpm test --compiler gfortran --profile release -``` +The remainder of this section contains commands for building Fiats with a recent Fortran compiler and the Fortran Package Manager ([`fpm`]). -#### NAG (`nagfor`) +#### LLVM (`flang-new`) +With LLVM `flang` 20 installed in your `PATH`, build and test Fiats with the installed `flang-new` symlink in order for `fpm` to correctly identify the compiler: ``` -fpm test --compiler nagfor --flag -fpp --profile release +fpm test --compiler flang-new --flag "-O3" ``` +With LLVM `flang` 19, enable the compiler's experimental support for assumed-rank entities: -#### LLVM (`flang-new`) -Building with `flang-new` requires passing flags to enable the compiler's experimental support for assumed-rank entities: ``` fpm test --compiler flang-new --flag "-mmlir -allow-assumed-rank -O3" ``` ##### _Experimental:_ Automatic parallelization of `do concurrent` on CPUs -With the `amd_trunk_dev` branch of the [ROCm fork] fork of LLVM, automatic parallelization currently works for inference, e.g. +With the `amd_trunk_dev` branch of the [ROCm fork] fork of LLVM, automatically parallelize inference calculations inside `do concurrent` constructs: ``` fpm run \ --example concurrent-inferences \ @@ -89,10 +82,25 @@ fpm run \ -- --network model.json ``` -where `model.json` must be a neural network in the [JSON] format used by Inference-Engine and the companion [nexport] package. -Automatic parallelization for training is under development. +where `model.json` must be a neural network in the [JSON] format used by Fiats and the companion [nexport] package. + +Automatic parallelization for training neural networks is under development. + +#### NAG (`nagfor`) +``` +fpm test --compiler nagfor --flag -fpp --profile release +``` + +#### GNU (`gfortran`) +Compiler bugs related to parameterized derived types currently prevent `gfortran` from building Fiats versions 0.15.0 or later. +Test and build earlier versions of Fiats build with the following command: +``` +fpm test --compiler gfortran --profile release +``` #### Intel (`ifx`) +Compiler bugs related to parameterized derived types currently prevent `gfortran` from building Fiats versions 0.15.0 or later. +Test and build earlier versions of Fiats build with the following command: ``` fpm test --compiler ifx --profile release --flag -O3 ``` @@ -100,10 +108,10 @@ fpm test --compiler ifx --profile release --flag -O3 ##### _Experimental:_ Automatic offloading of `do concurrent` to GPUs This capability is under development with the goal to facilitate automatic GPU offloading via the following command: ``` -fpm test --compiler ifx --profile releae --flag "-fopenmp-target-do-concurrent -qopenmp -fopenmp-targets=spir64 -O3" +fpm test --compiler ifx --profile release --flag "-fopenmp-target-do-concurrent -qopenmp -fopenmp-targets=spir64 -O3" ``` -#### HPE (`crayftn.sh`) -- under development +#### HPE (`crayftn.sh`) Support for the Cray Compiler Environment (CCE) Fortran compiler is under development. Building with the CCE `ftn` compiler wrapper requires an additional trivial wrapper shell script. For example, create a file `crayftn.sh` with the following contents and @@ -119,7 +127,7 @@ fpm test --compiler crayftn.sh ``` ### Configuring a training run -Inference-Engine imports hyperparameters and network configurations to and from JSON files. +Fiats imports hyperparameters and network configurations to and from JSON files. To see the expected file format, run the [print-training-configuration] example as follows: ``` % fpm run --example print-training-configuration --compiler gfortran @@ -141,7 +149,7 @@ Project is up to date } } ``` -Inference-Engine's JSON file format is fragile: splitting or combining lines breaks the file reader. +The Fiats JSON file format is fragile: splitting or combining lines breaks the file reader. Files with added or removed white space or reordered whole objects ("hyperparameters" or "network configuration") should work. A future release will leverage the [rojff] JSON interface to allow for more flexible file formatting. @@ -158,35 +166,42 @@ The following is representative output after 3000 epochs: 2000 0.61259E-04 9.8345 2,4,72,2,1 3000 0.45270E-04 14.864 2,4,72,2,1 ``` -The example program halts execution after reaching a cost-function threshold (which requires millions of epochws) or a maximum number of iterations or if the program detects a file named `stop` in the source-tree root directory. +The example program halts execution after reaching a cost-function threshold (which requires millions of epochs) or a maximum number of iterations or if the program detects a file named `stop` in the source-tree root directory. Before halting, the program will print a table of expected and predicted saturated mixing ratio values across a range of input pressures and temperatures, wherein two the inputs have each been mapped to the unit interval [0,1]. The program also writes the neural network initial condition to `initial-network.json` and the final (trained) network to the file specified in the above command: `sat-mix-rat.json`. ### Performing inference -Users with a PyTorch model may use [nexport] to export the model to JSON files that Inference-Engine can read. +Users with a PyTorch model may use [nexport] to export the model to JSON files that Fiats can read. Examples of performing inference using a neural-network JSON file are in [example/concurrent-inferences]. Documentation ------------- -Please see our [GitHub Pages site] for HTML documentation generated by [`ford`] or generate documentaiton locally by installing `ford` and executing `ford ford.md`. +### HTML +Please see our [GitHub Pages site] for Hypertext Markup Languge (HTML) documentation generated by [`ford`] or generate documentation locally by installing `ford` and executing `ford ford.md`. +### UML +Please see the `doc/uml` subdirectory for Unified Modeling Language (UML) diagrams such as a comprehensive Fiats [class diagram] with human-readable [Mermaid] source that renders graphically when opened by browsing to the document on GitHub. [Building and testing]: #building-and-testing [Caffeine]: https://go.lbl.gov/caffeine +[class diagram]: ./doc/uml/class-diagram.md +[Documentation]: #documentation +[demo]: demo [E3SM]: https://e3sm.org [example]: example -[demo]: demo -[Documentation]: #documentation [example/print-training-configuration.F90]: example/print-training-configuration.F90 [example/concurrent-inferences]: example/concurrent-inferences.f90 [`ford`]: https://github.com/Fortran-FOSS-Programmers/ford [`fpm`]: https://github.com/fortran-lang/fpm [Getting Started]: #getting-started -[GitHub Pages site]: https://berkeleylab.github.io/inference-engine/ +[GitHub Pages site]: https://berkeleylab.github.io/fiats/ [handy-dandy]: https://github.com/rouson/handy-dandy/blob/main/src +[HDF5]: https://www.hdfgroup.org/solutions/hdf5/ [ICAR]: https://github.com/BerkeleyLab/icar/tree/neural-net [JSON]: https://www.json.org/json-en.html [LLVM]: https://github.com/llvm/llvm-project +[Mermaid]: https://mermaid.js.org +[NetCDF]: https://www.unidata.ucar.edu/software/netcdf/ [nexport]: https://go.lbl.gov/nexport [Overview]: #overview [ROCm fork]: https://github.com/ROCm/llvm-project diff --git a/demo/app/infer-aerosol.f90 b/demo/app/infer-aerosol.f90 index d2211f2e9..d052f8481 100644 --- a/demo/app/infer-aerosol.f90 +++ b/demo/app/infer-aerosol.f90 @@ -7,7 +7,7 @@ program infer_aerosol use iso_fortran_env, only : int64, real64 ! External dependencies: - use inference_engine_m, only : unmapped_network_t, tensor_t, double_precision, double_precision_file_t + use fiats_m, only : unmapped_network_t, tensor_t, double_precision, double_precision_file_t use julienne_m, only : string_t, command_line_t use omp_lib @@ -49,7 +49,7 @@ subroutine read_stats_and_perform_inference(path) double precision cube_root double precision, allocatable, dimension(:,:) :: aerosol_data, input_components, output_components type(tensor_statistics_t) input_stats, output_stats - type(unmapped_network_t(double_precision)) inference_engine + type(unmapped_network_t(double_precision)) neural_network integer i, j input_stats = read_tensor_statistics(path // "meanxp.txt", path // "stdxp.txt", num_inputs) !for pre-processing normalization @@ -76,7 +76,7 @@ subroutine read_stats_and_perform_inference(path) !$omp end parallel do print *, "Reading the neural network from " // network_file_name - inference_engine = unmapped_network_t(double_precision_file_t(path // network_file_name)) + neural_network = unmapped_network_t(double_precision_file_t(path // network_file_name)) time_inference: & block @@ -103,7 +103,7 @@ subroutine read_stats_and_perform_inference(path) !$ start_time = omp_get_wtime() !$omp parallel do shared(inputs,outputs,icc) do i = 1,icc - outputs(i) = inference_engine%infer(inputs(i)) + outputs(i) = neural_network%infer(inputs(i)) end do !$omp end parallel do !$ end_time = omp_get_wtime() diff --git a/demo/app/tensor-statistics.f90 b/demo/app/tensor-statistics.f90 index 3a216205a..8c5cf2d06 100644 --- a/demo/app/tensor-statistics.f90 +++ b/demo/app/tensor-statistics.f90 @@ -9,7 +9,7 @@ program tensor_statistics ! External dependencies: use julienne_m, only : command_line_t, file_t, string_t use assert_m, only : assert, intrinsic_array_t - use inference_engine_m, only : ubounds_t + use fiats_m, only : ubounds_t use ieee_arithmetic, only : ieee_is_nan use iso_fortran_env, only : int64, real64 diff --git a/demo/app/train-cloud-microphysics.F90 b/demo/app/train-cloud-microphysics.F90 index 9a8c5efcc..1187b42e7 100644 --- a/demo/app/train-cloud-microphysics.F90 +++ b/demo/app/train-cloud-microphysics.F90 @@ -12,10 +12,10 @@ program train_on_flat_distribution !! External dependencies: use julienne_m, only : string_t, file_t, command_line_t, bin_t use assert_m, only : assert, intrinsic_array_t - use inference_engine_m, only : & - inference_engine_t, mini_batch_t, input_output_pair_t, tensor_t, trainable_network_t, tensor_map_t, & - training_configuration_t, shuffle - + use fiats_m, only : & + neural_network_t, mini_batch_t, input_output_pair_t, tensor_t, trainable_network_t, tensor_map_t, training_configuration_t, & + shuffle + !! Internal dependencies: use phase_space_bin_m, only : phase_space_bin_t use NetCDF_file_m, only: NetCDF_file_t @@ -319,7 +319,7 @@ subroutine read_train_write(training_configuration, args, plot_file) read_or_initialize_engine: & if (io_status==0) then print *,"Reading network from file " // network_file - trainable_network = trainable_network_t(inference_engine_t(file_t(string_t(network_file)))) + trainable_network = trainable_network_t(neural_network_t(file_t(string_t(network_file)))) close(network_unit) else close(network_unit) diff --git a/demo/fpm.toml b/demo/fpm.toml index eefd579dd..9bed6f611 100644 --- a/demo/fpm.toml +++ b/demo/fpm.toml @@ -1,10 +1,10 @@ -name = "Inference-Engine-Demonstration-Applications" -license = "(Please see inference-engine/LICENSE.txt.)" -author = "(Please see inference-engine/fpm.toml.)" -maintainer = "(Please see inference-engine/fpm.toml.)" +name = "Fiats-Demonstration-Applications" +license = "(Please see fiats/LICENSE.txt.)" +author = "(Please see fiats/fpm.toml.)" +maintainer = "(Please see fiats/fpm.toml.)" [dependencies] assert = {git = "https://github.com/sourceryinstitute/assert", tag = "1.7.0"} julienne = {git = "https://github.com/berkeleylab/julienne", tag = "1.2.2"} -inference-engine = {path = "../"} +fiats = {path = "../"} netcdf-interfaces = {git = "https://github.com/LKedward/netcdf-interfaces.git", rev = "d2bbb71ac52b4e346b62572b1ca1620134481096"} diff --git a/demo/include b/demo/include new file mode 120000 index 000000000..f5030fe88 --- /dev/null +++ b/demo/include @@ -0,0 +1 @@ +../include \ No newline at end of file diff --git a/demo/setup.sh b/demo/setup.sh index 5d848cfe8..46ce5b05d 100755 --- a/demo/setup.sh +++ b/demo/setup.sh @@ -4,7 +4,7 @@ set -e # exit on error usage() { - echo "Inference Engine Setup Script" + echo "Fiats Demonstration Applications Setup Script" echo "" echo "USAGE:" echo "./setup.sh [--help|-h] | [-p|--prefix=PREFIX]" @@ -112,25 +112,25 @@ else PKG_CONFIG_PATH=`realpath "$PKG_CONFIG_PATH"` fi -INFERENCE_ENGINE_VERSION=$(grep version ../fpm.toml | grep -o '".*"' - | sed 's/"//g') +FIATS_VERSION=$(grep version ../fpm.toml | grep -o '".*"' - | sed 's/"//g') -INFERENCE_ENGINE_PC="$PKG_CONFIG_PATH/inference-engine.pc" -echo "INFERENCE_ENGINE_FPM_CC=\"$FPM_CC\"" > $INFERENCE_ENGINE_PC -echo "INFERENCE_ENGINE_FPM_FC=\"$FPM_FC\"" >> $INFERENCE_ENGINE_PC +FIATS_PC="$PKG_CONFIG_PATH/fiats.pc" +echo "FIATS_FPM_CC=\"$FPM_CC\"" > $FIATS_PC +echo "FIATS_FPM_FC=\"$FPM_FC\"" >> $FIATS_PC if [[ ! -z ${FPM_RUNNER:-} ]]; then - echo "INFERENCE_ENGINE_FPM_RUNNER=\"$FPM_RUNNER\"" >> $INFERENCE_ENGINE_PC + echo "FIATS_FPM_RUNNER=\"$FPM_RUNNER\"" >> $FIATS_PC fi -echo "INFERENCE_ENGINE_FPM_LD_FLAG=\"$FPM_LD_FLAG\"" >> $INFERENCE_ENGINE_PC -echo "INFERENCE_ENGINE_FPM_FLAG=\"$FPM_FLAG\"" >> $INFERENCE_ENGINE_PC -echo "Name: inference-engine" >> $INFERENCE_ENGINE_PC -echo "Description: Inference Engine" >> $INFERENCE_ENGINE_PC -echo "URL: https://github.com/berkeleylab/inference-engine" >> $INFERENCE_ENGINE_PC -echo "Version: $INFERENCE_ENGINE_VERSION" >> $INFERENCE_ENGINE_PC +echo "FIATS_FPM_LD_FLAG=\"$FPM_LD_FLAG\"" >> $FIATS_PC +echo "FIATS_FPM_FLAG=\"$FPM_FLAG\"" >> $FIATS_PC +echo "Name: fiats" >> $FIATS_PC +echo "Description: Fiats" >> $FIATS_PC +echo "URL: https://github.com/berkeleylab/fiats " >> $FIATS_PC +echo "Version: $FIATS_VERSION" >> $FIATS_PC if [ $CI = true ]; then echo "---------------" - echo "cat \$INFERENCE_ENGINE_PC" - cat $INFERENCE_ENGINE_PC + echo "cat \$FIATS_PC" + cat $FIATS_PC echo "---------------" fi @@ -139,14 +139,14 @@ cp src/run-fpm.sh-header build/run-fpm.sh RUN_FPM_SH="`realpath ./build/run-fpm.sh`" echo "`which fpm` \$fpm_arguments \\" >> $RUN_FPM_SH echo "--profile release \\" >> $RUN_FPM_SH -echo "--c-compiler \"`pkg-config inference-engine --variable=INFERENCE_ENGINE_FPM_CC`\" \\" >> $RUN_FPM_SH -echo "--compiler \"`pkg-config inference-engine --variable=INFERENCE_ENGINE_FPM_FC`\" \\" >> $RUN_FPM_SH +echo "--c-compiler \"`pkg-config fiats --variable=FIATS_FPM_CC`\" \\" >> $RUN_FPM_SH +echo "--compiler \"`pkg-config fiats --variable=FIATS_FPM_FC`\" \\" >> $RUN_FPM_SH if [[ ! -z ${FPM_RUNNER:-} ]]; then - echo "--runner \"`pkg-config inference-engine --variable=INFERENCE_ENGINE_FPM_RUNNER`\" \\" >> $RUN_FPM_SH + echo "--runner \"`pkg-config fiats --variable=FIATS_FPM_RUNNER`\" \\" >> $RUN_FPM_SH fi -echo "--flag \"-cpp `pkg-config inference-engine --variable=INFERENCE_ENGINE_FPM_FLAG`\" \\" >> $RUN_FPM_SH -echo "--link-flag \"`pkg-config inference-engine --variable=INFERENCE_ENGINE_FPM_LD_FLAG`\" \\" >> $RUN_FPM_SH -echo "\$program_arguments" >> $RUN_FPM_SH +echo "--flag \"-cpp `pkg-config fiats --variable=FIATS_FPM_FLAG`\" \\" >> $RUN_FPM_SH +echo "--link-flag \"`pkg-config fiats --variable=FIATS_FPM_LD_FLAG`\" \\" >> $RUN_FPM_SH +echo "\$program_arguments" >> $RUN_FPM_SH chmod u+x $RUN_FPM_SH if [ $CI = true ]; then echo "---------------" @@ -158,10 +158,10 @@ fi $RUN_FPM_SH build echo "" -echo "____________________ The inference-engine demo apps build succeeded! _______________________" +echo "____________________ The fiats demo apps build succeeded! _______________________" echo "" echo "Run the following command to see a list of available apps:" echo "" echo "./build/run-fpm.sh run" echo "" -echo "Append a space followe by an app's name to see basic app usage information." +echo "Append a space followed by an app's name to see basic app usage information." diff --git a/doc/uml/class-diagram.md b/doc/uml/class-diagram.md new file mode 100644 index 000000000..2402c09ba --- /dev/null +++ b/doc/uml/class-diagram.md @@ -0,0 +1,167 @@ +Fiats Class Diagram +------------------- +```mermaid +classDiagram + +class activation_t +class double_precision_string_t +class metadata_t +class mini_batch_t +class neural_network_t +class string_t +class tensor_t +class tensor_map_t +class trainable_network_t +class workspace_t + +neural_network_t <|-- trainable_network_t +neural_network_t o-- metadata_t +neural_network_t o-- tensor_map_t +neural_network_t o-- activation_t +trainable_network_t o-- workspace_t +string_t <|-- double_precision_string_t + +class double_precision_file_t +class file_t +class hyperparameters_t +class layer_t +class network_configuration_t +class neuron_t + +mini_batch_t o--"1..*" input_output_pair_t +input_output_pair_t o-- "2" tensor_t +file_t <|-- double_precision_file_t + +layer_t o-- neuron_t +layer_t o--"0..*" layer_t +neuron_t o--"0..*" neuron_t +training_configuration_t o-- hyperparameters_t +training_configuration_t o-- network_configuration_t + +class neural_network_t{ + - input_map_ : tensor_map_t + - output_map_ : tensor_map_t + - activation_ : activation_t + - metadata_ : metadata_t + - weights_ : real + - bisases_ : real + - nodes_ : integer + + operator(==) logical + + infer(tensor_t) tensor_t + + to_json() file_t + + map_to_input_range(tensor_t) tensor_t + + map_from_output_range(tensor_t) tensor_t + + num_hidden_layers() integer + + num_inputs() integer + + num_outputs() integer + + nodes_per_layer() integer + + assert_conformable_with() + + skip() logical + + activation_function_name() string_t + + learn(mini_batch_t, real, logical, real, workspace_t) + + assert_consistency() +} +class trainable_network_t{ + - workspace_ : workspace_t + + train(mini_batch_t, real, logical, real) + + map_to_training_ranges() : input_output_pair_t +} +class tensor_map_t{ + - layer_ : character + - intercept_ : real + - slope_ : real + + to_json() : file_t + + operator(==) logical +} +class tensor_t{ + - values_ : real + + values() real + - num_components() integer +} +class metadata_t{ + - modelName_ : string_t + - modelAuthor_ : string_t + - compilationDate_ : string_t + - activationFunction_: string_t + - usingSkipConnections_ : string_t + + strings() string_t + + to_json() file_t + + activation_name() string_t + + operator(==) logical +} +class mini_batch_t{ + - input_output_pairs_ : input_output_pair_t + + input_output_pairs() input_output_pair_t +} +class activation_t{ + - selection_ : integer + + function_name() string_t + + evaluate() real + + differentiate() real +} +class input_output_pair_t{ + - inputs_ : tensor_t + - expected_outputs_ : tensor_t + + inputs() tensor_t + + expected_outputs() tensor_t + + shuffle() + + write_to_stdout() +} +class file_t{ + - lines_ : string_t + + lines() : string_t + + write_lines(string_t) +} +class double_precision_file_t{ + + double_precision_lines() double_precision_string_t +} +class hyperparameters_t{ + - mini_batches_ : integer + - learning_rate_ : real + - optimizer_ : character + + to_json() file_t + + mini_batches() : integer + + learning_rate() : real + + optimizer_name() : character +} +class network_configuration_t{ + - skip_connections_ : logical + - nodes_per_layer_: integer + - activation_name_ : character + + to_json() file_t + + operator(==) logical + + activation_name() string_t + + nodes_per_layer() integer + + skip_connections() logical +} +class layer_t{ + - neuron_ : neuron_t + - next_ : layer_t + + neural_network() : neural_network_t + + count_layers() integer + + count_neurons() integer + + count_inputs() integer + + neurons_per_layer() integer + + next_allocated() logical + + next_pointer() layer_t +} +class neuron_t{ + - weights_ : real + - bias_ : next + + to_json() file_t + + weights() real + + bias() real + + next_allocated() logical + + next_pointer() neuron_t + + num_inputs() integer +} +class training_configuration_t{ + - hyperparameters_ : hyperparameters_t + - network_configuration : nework_configuration_t + + operator(==) logical + + to_json() file_t + + mini_batches() integer + + optimizer_name() string_t + + nodes_per_layer() integer + + skip_connections() logical +} diff --git a/doc/uml_diagrams/inference_engine.md b/doc/uml_diagrams/inference_engine.md deleted file mode 100644 index 6d5b7bd39..000000000 --- a/doc/uml_diagrams/inference_engine.md +++ /dev/null @@ -1,52 +0,0 @@ -inference engine ---- -```mermaid -classDiagram - -class activation_strategy_t - -activation_strategy_t <|-- differentiable_activation_strategy_t - -class expected_outputs_t - -class inference_engine_t - -inference_engine_t o-- string_t - -input_output_pair_t o-- inputs_t -input_output_pair_t o-- expected_outputs_t - -layer_t o-- neuron_t -layer_t o--"0..*" layer_t - -mini_batch_t o--"0..*" input_output_pair_t - -class netCDF_file_t - -neuron_t o--"0..*" neuron_t - -class outputs_t - - -differentiable_activation_strategy_t <|--sigmoid_t -<> differentiable_activation_strategy_t -activation_strategy_t <|-- step_t - -differentiable_activation_strategy_t <|-- swish_t - -class trainable_engine_t{ - -metadata_:string - -w : real - -b : real - -n : integer - - +assert_consistent() - +train() - +infer() - +num_layers() - +num_inputs() - +to_inference_engine() - differentiable_activation_strategy_t() - -} -trainable_engine_t o--"0..*" string_t diff --git a/example/concurrent-inferences.f90 b/example/concurrent-inferences.f90 index 5036e67b7..9a50ae249 100644 --- a/example/concurrent-inferences.f90 +++ b/example/concurrent-inferences.f90 @@ -3,7 +3,7 @@ program concurrent_inferences !! This program demonstrates how to read a neural network from a JSON file !! and use the network to perform concurrent inferences. - use inference_engine_m, only : inference_engine_t, tensor_t, double_precision, double_precision_string_t, double_precision_file_t + use fiats_m, only : neural_network_t, tensor_t, double_precision, double_precision_string_t, double_precision_file_t use julienne_m, only : string_t, command_line_t, file_t use assert_m, only : assert use iso_fortran_env, only : int64, real64 @@ -27,17 +27,17 @@ program concurrent_inferences block integer(int64) t_start, t_finish, clock_rate - type(inference_engine_t) inference_engine + type(neural_network_t) neural_network type(tensor_t), allocatable :: inputs(:,:,:), outputs(:,:,:) real, allocatable :: input_components(:,:,:,:) - print *, "Constructing a new inference_engine_t object from the file " // network_file_name%string() - inference_engine = inference_engine_t(file_t(network_file_name)) + print *, "Constructing a new neural_network_t object from the file " // network_file_name%string() + neural_network = neural_network_t(file_t(network_file_name)) print *,"Defining an array of tensor_t input objects with random normalized components" allocate(outputs(lat,lon,lev)) allocate( inputs(lat,lon,lev)) - allocate(input_components(lat,lon,lev,inference_engine%num_inputs())) + allocate(input_components(lat,lon,lev,neural_network%num_inputs())) call random_number(input_components) do concurrent(i=1:lat, j=1:lon, k=1:lev) @@ -47,7 +47,7 @@ program concurrent_inferences print *,"Performing concurrent inference" call system_clock(t_start, clock_rate) do concurrent(i=1:lat, j=1:lon, k=1:lev) - outputs(i,j,k) = inference_engine%infer(inputs(i,j,k)) + outputs(i,j,k) = neural_network%infer(inputs(i,j,k)) end do call system_clock(t_finish) print *,"Concurrent inference time: ", real(t_finish - t_start, real64)/real(clock_rate, real64) @@ -57,7 +57,7 @@ program concurrent_inferences do k=1,lev do j=1,lon do i=1,lat - outputs(i,j,k) = inference_engine%infer(inputs(i,j,k)) + outputs(i,j,k) = neural_network%infer(inputs(i,j,k)) end do end do end do @@ -66,7 +66,7 @@ program concurrent_inferences print *,"Performing elemental inferences" call system_clock(t_start, clock_rate) - outputs = inference_engine%infer(inputs) ! implicit (re)allocation of outputs array only if shape(inputs) /= shape(outputs) + outputs = neural_network%infer(inputs) ! implicit (re)allocation of outputs array only if shape(inputs) /= shape(outputs) call system_clock(t_finish) print *,"Elemental inference time: ", real(t_finish - t_start, real64)/real(clock_rate, real64) @@ -76,17 +76,17 @@ program concurrent_inferences block integer(int64) t_start, t_finish, clock_rate - type(inference_engine_t(double_precision)) inference_engine + type(neural_network_t(double_precision)) neural_network type(tensor_t(double_precision)), allocatable :: inputs(:,:,:), outputs(:,:,:) double precision, allocatable :: input_components(:,:,:,:) - print *, "Constructing a new inference_engine_t object from the file " // network_file_name%string() - inference_engine = inference_engine_t(double_precision_file_t(network_file_name)) + print *, "Constructing a new neural_network_t object from the file " // network_file_name%string() + neural_network = neural_network_t(double_precision_file_t(network_file_name)) print *,"Defining an array of tensor_t input objects with random normalized components" allocate(outputs(lat,lon,lev)) allocate( inputs(lat,lon,lev)) - allocate(input_components(lat,lon,lev,inference_engine%num_inputs())) + allocate(input_components(lat,lon,lev,neural_network%num_inputs())) call random_number(input_components) do concurrent(i=1:lat, j=1:lon, k=1:lev) @@ -96,7 +96,7 @@ program concurrent_inferences print *,"Performing double-precision concurrent inference" call system_clock(t_start, clock_rate) do concurrent(i=1:lat, j=1:lon, k=1:lev) - outputs(i,j,k) = inference_engine%infer(inputs(i,j,k)) + outputs(i,j,k) = neural_network%infer(inputs(i,j,k)) end do call system_clock(t_finish) print *,"Double-precision concurrent inference time: ", real(t_finish - t_start, real64)/real(clock_rate, real64) diff --git a/example/learn-addition.F90 b/example/learn-addition.F90 index b6174650f..a5ca64ffa 100644 --- a/example/learn-addition.F90 +++ b/example/learn-addition.F90 @@ -2,7 +2,7 @@ ! Terms of use are as specified in LICENSE.txt module addition_m !! Define a function that produces the desired network output for a given network input - use inference_engine_m, only : tensor_t + use fiats_m, only : tensor_t use assert_m, only : assert implicit none @@ -20,8 +20,8 @@ elemental function y(x_tensor) result(a_tensor) program learn_addition !! This trains a neural network to learn the following six polynomial functions of its eight inputs. - use inference_engine_m, only : & - inference_engine_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle + use fiats_m, only : & + neural_network_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle use julienne_m, only : string_t, file_t, command_line_t, bin_t use assert_m, only : assert, intrinsic_array_t use addition_m, only : y @@ -107,11 +107,11 @@ program learn_addition contains - subroutine output(inference_engine, file_name) - class(inference_engine_t), intent(in) :: inference_engine + subroutine output(neural_network, file_name) + class(neural_network_t), intent(in) :: neural_network type(string_t), intent(in) :: file_name type(file_t) json_file - json_file = inference_engine%to_json() + json_file = neural_network%to_json() call json_file%write_lines(file_name) end subroutine @@ -140,7 +140,7 @@ function perturbed_identity_network(perturbation_magnitude) result(trainable_net associate(w => identity + perturbation_magnitude*(w_harvest-0.5)/0.5, b => perturbation_magnitude*(b_harvest-0.5)/0.5) - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = n, weights = w, biases = b, metadata = & [string_t("Perturbed Identity"), string_t("Damian Rouson"), string_t("2023-09-23"), string_t("relu"), string_t("false")] & )) diff --git a/example/learn-exponentiation.F90 b/example/learn-exponentiation.F90 index a09b9359d..cfb549b85 100644 --- a/example/learn-exponentiation.F90 +++ b/example/learn-exponentiation.F90 @@ -2,7 +2,7 @@ ! Terms of use are as specified in LICENSE.txt module exponentiation_m !! Define a function that produces the desired network output for a given network input - use inference_engine_m, only : tensor_t + use fiats_m, only : tensor_t use assert_m, only : assert implicit none @@ -20,8 +20,7 @@ elemental function y(x_tensor) result(a_tensor) program learn_exponentiation !! This trains a neural network to learn the following six polynomial functions of its eight inputs. - use inference_engine_m, only : & - inference_engine_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle + use fiats_m, only : neural_network_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle use julienne_m, only : string_t, file_t, command_line_t, bin_t use assert_m, only : assert, intrinsic_array_t use exponentiation_m, only : y @@ -107,11 +106,11 @@ program learn_exponentiation contains - subroutine output(inference_engine, file_name) - class(inference_engine_t), intent(in) :: inference_engine + subroutine output(neural_network, file_name) + class(neural_network_t), intent(in) :: neural_network type(string_t), intent(in) :: file_name type(file_t) json_file - json_file = inference_engine%to_json() + json_file = neural_network%to_json() call json_file%write_lines(file_name) end subroutine @@ -140,7 +139,7 @@ function perturbed_identity_network(perturbation_magnitude) result(trainable_net associate(w => identity + perturbation_magnitude*(w_harvest-0.5)/0.5, b => perturbation_magnitude*(b_harvest-0.5)/0.5) - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = n, weights = w, biases = b, metadata = & [string_t("Perturbed Identity"), string_t("Damian Rouson"), string_t("2023-09-23"), string_t("relu"), string_t("false")] & )) diff --git a/example/learn-multiplication.F90 b/example/learn-multiplication.F90 index 81d72e88d..116c961c0 100644 --- a/example/learn-multiplication.F90 +++ b/example/learn-multiplication.F90 @@ -2,7 +2,7 @@ ! Terms of use are as specified in LICENSE.txt module multiply_inputs !! Define a function that produces the desired network output for a given network input - use inference_engine_m, only : tensor_t + use fiats_m, only : tensor_t use assert_m, only : assert implicit none @@ -20,8 +20,7 @@ elemental function y(x_tensor) result(a_tensor) program learn_multiplication !! This trains a neural network to learn the following six polynomial functions of its eight inputs. - use inference_engine_m, only : & - inference_engine_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle + use fiats_m, only : neural_network_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle use julienne_m, only : string_t, file_t, command_line_t, bin_t use assert_m, only : assert, intrinsic_array_t use multiply_inputs, only : y @@ -107,11 +106,11 @@ program learn_multiplication contains - subroutine output(inference_engine, file_name) - class(inference_engine_t), intent(in) :: inference_engine + subroutine output(neural_network, file_name) + class(neural_network_t), intent(in) :: neural_network type(string_t), intent(in) :: file_name type(file_t) json_file - json_file = inference_engine%to_json() + json_file = neural_network%to_json() call json_file%write_lines(file_name) end subroutine @@ -140,7 +139,7 @@ function perturbed_identity_network(perturbation_magnitude) result(trainable_net associate(w => identity + perturbation_magnitude*(w_harvest-0.5)/0.5, b => perturbation_magnitude*(b_harvest-0.5)/0.5) - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = n, weights = w, biases = b, metadata = & [string_t("Perturbed Identity"), string_t("Damian Rouson"), string_t("2023-09-23"), string_t("relu"), string_t("false")] & )) diff --git a/example/learn-power-series.F90 b/example/learn-power-series.F90 index e8a46e6dc..3cbcc2719 100644 --- a/example/learn-power-series.F90 +++ b/example/learn-power-series.F90 @@ -2,7 +2,7 @@ ! Terms of use are as specified in LICENSE.txt module power_series !! Define a function that produces the desired network output for a given network input - use inference_engine_m, only : tensor_t + use fiats_m, only : tensor_t use assert_m, only : assert implicit none @@ -20,8 +20,7 @@ elemental function y(x_in) result(a) program learn_power_series !! This trains a neural network to learn the following six polynomial functions of its eight inputs. - use inference_engine_m, only : & - inference_engine_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle + use fiats_m, only : neural_network_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle use julienne_m, only : string_t, file_t, command_line_t, bin_t use assert_m, only : assert, intrinsic_array_t use power_series, only : y @@ -109,11 +108,11 @@ program learn_power_series contains - subroutine output(inference_engine, file_name) - class(inference_engine_t), intent(in) :: inference_engine + subroutine output(neural_network, file_name) + class(neural_network_t), intent(in) :: neural_network type(string_t), intent(in) :: file_name type(file_t) json_file - json_file = inference_engine%to_json() + json_file = neural_network%to_json() call json_file%write_lines(file_name) end subroutine @@ -142,7 +141,7 @@ function perturbed_identity_network(perturbation_magnitude) result(trainable_net associate(w => identity + perturbation_magnitude*(w_harvest-0.5)/0.5, b => perturbation_magnitude*(b_harvest-0.5)/0.5) - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = n, weights = w, biases = b, metadata = & [string_t("Perturbed Identity"), string_t("Damian Rouson"), string_t("2023-09-23"), string_t("relu"), string_t("false")] & )) diff --git a/example/learn-saturated-mixing-ratio.F90 b/example/learn-saturated-mixing-ratio.F90 index bf49da58c..29313e2c7 100644 --- a/example/learn-saturated-mixing-ratio.F90 +++ b/example/learn-saturated-mixing-ratio.F90 @@ -2,8 +2,7 @@ ! Terms of use are as specified in LICENSE.txt program train_saturated_mixture_ratio !! This program trains a neural network to learn the saturated mixing ratio function of ICAR. - use inference_engine_m, only : & - inference_engine_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle + use fiats_m, only : neural_network_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle use julienne_m, only : string_t, file_t, command_line_t, bin_t, csv use assert_m, only : assert, intrinsic_array_t use saturated_mixing_ratio_m, only : y, T, p @@ -43,7 +42,7 @@ program train_saturated_mixture_ratio if (io_status == io_success) then print *,"Reading network from file " // network_file%string() - trainable_network = trainable_network_t(inference_engine_t(file_t(network_file))) + trainable_network = trainable_network_t(neural_network_t(file_t(network_file))) close(network_unit) else close(network_unit) @@ -155,11 +154,11 @@ subroutine print_diagnostics(plot_file_unit, epoch, cost, clock, nodes) write(unit=plot_file_unit, fmt=csv) nodes end subroutine - subroutine output(inference_engine, file_name) - class(inference_engine_t), intent(in) :: inference_engine + subroutine output(neural_network, file_name) + class(neural_network_t), intent(in) :: neural_network type(string_t), intent(in) :: file_name type(file_t) json_file - json_file = inference_engine%to_json() + json_file = neural_network%to_json() call json_file%write_lines(file_name) end subroutine @@ -187,7 +186,7 @@ function perturbed_identity_network(perturbation_magnitude, n) result(trainable_ call random_number(b_harvest) associate(w => identity + perturbation_magnitude*(w_harvest-0.5)/0.5, b => perturbation_magnitude*(b_harvest-0.5)/0.5) - trainable_network = trainable_network_t( inference_engine_t(nodes=n, weights=w, biases=b, & + trainable_network = trainable_network_t( neural_network_t(nodes=n, weights=w, biases=b, & metadata=[string_t("Saturated Mixing Ratio"),string_t("Rouson"),string_t("20241013"),string_t("relu"),string_t("false")] & )) end associate diff --git a/example/print-training-configuration.F90 b/example/print-training-configuration.F90 index cf6f4996b..36ed0983f 100644 --- a/example/print-training-configuration.F90 +++ b/example/print-training-configuration.F90 @@ -2,7 +2,7 @@ ! Terms of use are as specified in LICENSE.txt program print_training_configuration !! Demonstrate how to construct and print a training_configuration_t object - use inference_engine_m, only : training_configuration_t, hyperparameters_t, network_configuration_t + use fiats_m, only : training_configuration_t, hyperparameters_t, network_configuration_t use julienne_m, only : file_t implicit none #ifdef _CRAYFTN diff --git a/example/read-query-infer.f90 b/example/read-query-infer.f90 index 86889192b..66bb7c7b0 100644 --- a/example/read-query-infer.f90 +++ b/example/read-query-infer.f90 @@ -4,7 +4,7 @@ program read_query_infer !! This program demonstrates how to read a neural network from a JSON file, !! query the network object for some of its properties, print those properties, !! and use the network to perform inference. - use inference_engine_m, only : inference_engine_t, tensor_t + use fiats_m, only : neural_network_t, tensor_t use julienne_m, only : string_t, command_line_t, file_t implicit none @@ -17,16 +17,16 @@ program read_query_infer 'Usage: fpm run --example read-query -- --input-file ""' end if - print *, "Reading an inference_engine_t object from the same JSON file '"//file_name%string()//"'." - associate(inference_engine => inference_engine_t(file_t(file_name))) + print *, "Reading an neural_network_t object from the same JSON file '"//file_name%string()//"'." + associate(neural_network => neural_network_t(file_t(file_name))) - print *, "Querying the new inference_engine_t object for several properties:" - associate(activation_name => inference_engine%activation_function_name()) + print *, "Querying the new neural_network_t object for several properties:" + associate(activation_name => neural_network%activation_function_name()) print *, "Activation function: ", activation_name%string() end associate - print *, "Number of outputs:", inference_engine%num_outputs() - print *, "Number of inputs:", inference_engine%num_inputs() - print *, "Nodes per layer:", inference_engine%nodes_per_layer() + print *, "Number of outputs:", neural_network%num_outputs() + print *, "Number of inputs:", neural_network%num_inputs() + print *, "Nodes per layer:", neural_network%nodes_per_layer() print *, "Performing inference:" block @@ -41,13 +41,13 @@ program read_query_infer do i = 1, num_tests call random_number(harvest) associate(inputs => tensor_t(tensor_min + (tensor_max-tensor_min)*harvest)) - associate(outputs => inference_engine%infer(inputs)) + associate(outputs => neural_network%infer(inputs)) print '(2(2g12.5,a,2x))', inputs%values(), "|", outputs%values() end associate end associate end do end block - end associate ! associate(inference_engine => ...) + end associate ! associate(neural_network => ...) end associate ! associate(file_name => ...) end program diff --git a/example/supporting-modules/saturated_mixing_ratio_m.f90 b/example/supporting-modules/saturated_mixing_ratio_m.f90 index e888f652d..88067358d 100644 --- a/example/supporting-modules/saturated_mixing_ratio_m.f90 +++ b/example/supporting-modules/saturated_mixing_ratio_m.f90 @@ -27,7 +27,7 @@ module saturated_mixing_ratio_m !! The saturated_mixing_ratio function in this module resulted from refactoring the sat_mr function !! in the Intermediate Complexity Atmospheric Research (ICAR) model file src/physics/mp_simple.f90. !! ICAR is distributed under the above MIT license. See https://github.com/ncar/icar. - use inference_engine_m, only : tensor_t + use fiats_m, only : tensor_t use assert_m, only : assert implicit none diff --git a/example/train-and-write.F90 b/example/train-and-write.F90 index 7105afde0..3594b1007 100644 --- a/example/train-and-write.F90 +++ b/example/train-and-write.F90 @@ -8,8 +8,8 @@ program train_and_write !! the desired network therefore contains weights corresponding to identity matrices and biases that vanish everywhere. !! The initial condition corresponds to the desired network with all weights and biases perturbed by a random variable !! that is uniformly distributed on the range [0,0.1]. - use inference_engine_m, only : & - inference_engine_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle + use fiats_m, only : & + neural_network_t, trainable_network_t, mini_batch_t, tensor_t, input_output_pair_t, shuffle use julienne_m, only : string_t, file_t, command_line_t, bin_t use assert_m, only : assert, intrinsic_array_t implicit none @@ -98,11 +98,11 @@ program train_and_write contains - subroutine output(inference_engine, file_name) - class(inference_engine_t), intent(in) :: inference_engine + subroutine output(neural_network, file_name) + class(neural_network_t), intent(in) :: neural_network type(string_t), intent(in) :: file_name type(file_t) json_file - json_file = inference_engine%to_json() + json_file = neural_network%to_json() call json_file%write_lines(file_name) end subroutine @@ -130,7 +130,7 @@ function perturbed_identity_network(perturbation_magnitude) result(trainable_net associate(w => identity + perturbation_magnitude*(w_harvest-0.5)/0.5, b => perturbation_magnitude*(b_harvest-0.5)/0.5) - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = nodes_per_layer, weights = w, biases = b, metadata = & [string_t("Perturbed Identity"), string_t("Damian Rouson"), string_t("2024-10-13"), string_t("relu"), string_t("false")] & )) diff --git a/example/write-read-infer.F90 b/example/write-read-infer.F90 index 8786d4b24..1f0cbed31 100644 --- a/example/write-read-infer.F90 +++ b/example/write-read-infer.F90 @@ -7,7 +7,7 @@ program write_read_infer !! perform inference. The network performs an identity mapping from any !! non-negative inputs to the corresponding outputs using a RELU activation !! function. - use inference_engine_m, only : inference_engine_t, tensor_t + use fiats_m, only : neural_network_t, tensor_t use julienne_m, only : string_t, command_line_t, file_t implicit none @@ -25,21 +25,21 @@ program write_read_infer contains - function identity_network() result(inference_engine) - type(inference_engine_t) inference_engine + function identity_network() result(neural_network) + type(neural_network_t) neural_network integer, parameter :: nodes_per_layer(*) = [2, 2, 2] integer, parameter :: max_n = maxval(nodes_per_layer), layers = size(nodes_per_layer) #ifdef _CRAYFTN real, allocatable :: weights(:,:,:) weights = reshape([real :: [1,0, 0,1], [1,0, 0,1]], [max_n, max_n, layers-1]) - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = [string_t("Identity"), string_t("Damian Rouson"), string_t("2023-09-18"), string_t("relu"), string_t("false")], & weights = weights, & biases = reshape([real:: [0,0], [0,0]], [max_n, layers-1]), & nodes = nodes_per_layer & ) #else - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = [string_t("Identity"), string_t("Damian Rouson"), string_t("2023-09-18"), string_t("relu"), string_t("false")], & weights = reshape([real :: [1,0, 0,1], [1,0, 0,1]], [max_n, max_n, layers-1]), & biases = reshape([real :: [0,0], [0,0]], [max_n, layers-1]), & @@ -53,35 +53,35 @@ subroutine write_read_query_infer(output_file_name) type(string_t) activation_name integer i, j integer, parameter :: num_neurons = 3, num_hidden_layers = 2 - type(inference_engine_t) network, inference_engine + type(neural_network_t) network, neural_network type(file_t) json_output_file, json_input_file type(tensor_t) inputs, outputs - print *, "Constructing an inference_engine_t neural-network object from scratch." + print *, "Constructing an neural_network_t neural-network object from scratch." network = identity_network() - print *, "Converting an inference_engine_t object to a file_t object." + print *, "Converting an neural_network_t object to a file_t object." json_output_file = network%to_json() - print *, "Writing an inference_engine_t object to the file '"//output_file_name%string()//"' in JSON format." + print *, "Writing an neural_network_t object to the file '"//output_file_name%string()//"' in JSON format." call json_output_file%write_lines(output_file_name) - print *, "Reading an inference_engine_t object from the same JSON file '"//output_file_name%string()//"'." + print *, "Reading an neural_network_t object from the same JSON file '"//output_file_name%string()//"'." json_input_file = file_t(output_file_name) - print *, "Constructing a new inference_engine_t object from the parameters read." - inference_engine = inference_engine_t(json_input_file) + print *, "Constructing a new neural_network_t object from the parameters read." + neural_network = neural_network_t(json_input_file) - print *, "Querying the new inference_engine_t object for several properties:" - print *, "Number of outputs:", inference_engine%num_outputs() - print *, "Number of inputs:", inference_engine%num_inputs() - print *, "Nodes per layer:", inference_engine%nodes_per_layer() - activation_name = inference_engine%activation_function_name() + print *, "Querying the new neural_network_t object for several properties:" + print *, "Number of outputs:", neural_network%num_outputs() + print *, "Number of inputs:", neural_network%num_inputs() + print *, "Nodes per layer:", neural_network%nodes_per_layer() + activation_name = neural_network%activation_function_name() print *, "Activation function: ", activation_name%string() print *, "Performing inference:" inputs = tensor_t([2.,3.]) print *, "Inputs: ", inputs%values() - outputs = inference_engine%infer(inputs) + outputs = neural_network%infer(inputs) print *, "Actual outputs: ", outputs%values() print *, "Correct outputs: ", inputs%values() end subroutine write_read_query_infer diff --git a/ford.md b/ford.md index 2e8c5c03e..35de3c8ef 100644 --- a/ford.md +++ b/ford.md @@ -1,5 +1,5 @@ -project: Inference-Engine -summary: A deep learning library targeting high-performance computing (HPC) applications with performance-critical inference and training needs. +project: Fiats +summary: Functional inference and training of surrogate models for computational science. src_dir: src/ src_dir: example exclude_dir: doc @@ -17,12 +17,12 @@ coloured_edges: true sort: permission-alpha extra_mods: iso_fortran_env:https://gcc.gnu.org/onlinedocs/gfortran/ISO_005fFORTRAN_005fENV.html iso_c_binding:https://gcc.gnu.org/onlinedocs/gfortran/ISO_005fC_005fBINDING.html#ISO_005fC_005fBINDING -project_github: https://github.com/berkeleylab/inference-engine +project_github: https://github.com/berkeleylab/fiats author: Berkeley Lab print_creation_date: true creation_date: %Y-%m-%d %H:%M %z -project_github: https://github.com/berkeleylab/inference-engine -project_download: https://github.com/berkeleylab/inference-engine/releases +project_github: https://github.com/berkeleylab/fiats +project_download: https://github.com/berkeleylab/fiats/releases github: https://github.com/berkeleylab predocmark_alt: > predocmark: < diff --git a/fpm.toml b/fpm.toml index 92ba67a53..815ffa443 100644 --- a/fpm.toml +++ b/fpm.toml @@ -1,11 +1,9 @@ -name = "inference-engine" -version = "0.13.1" +name = "fiats" +version = "0.15.0" license = "see LICENSE.txt" author = "Damian Rouson, Tan Nguyen, Jordan Welsman, David Torres, Brad Richardson, Katherine Rasmussen, Federica Villani, Dan Bonachea" maintainer = "rouson@lbl.gov" [dependencies] assert = {git = "https://github.com/sourceryinstitute/assert", tag = "1.7.0"} - -[dev-dependencies] julienne = {git = "https://github.com/berkeleylab/julienne", tag = "1.3.1"} diff --git a/setup.sh b/setup.sh index 5908ce704..455007afa 100755 --- a/setup.sh +++ b/setup.sh @@ -4,13 +4,13 @@ set -e # exit on error usage() { - echo "Inference Engine Setup Script" + echo "Fiats Setup Script" echo "" echo "USAGE:" echo "./setup.sh [--help|-h]" echo "" echo " --help Display this help text" - echo " --prefix=PREFIX Install any binaries needed to build inference-engine in 'PREFIX/bin'" + echo " --prefix=PREFIX Install any binaries needed to build fiats in 'PREFIX/bin'" echo " Default prefix='\$HOME/.local/bin'" echo "" } @@ -48,10 +48,10 @@ install_fpm_from_source() echo "Please install curl and then rerun ./setup.sh" exit 1 fi - mkdir temp-dir-to-build-fpm-for-inference-engine-installation - curl -L -o temp-dir-to-build-fpm-for-inference-engine-installation/fpm.F90 https://github.com/fortran-lang/fpm/releases/download/current/fpm.F90 - gfortran -o $PREFIX/bin/fpm -Jtemp-dir-to-build-fpm-for-inference-engine-installation temp-dir-to-build-fpm-for-inference-engine-installation/fpm.F90 - rm -rf temp-dir-to-build-fpm-for-inference-engine-installation + mkdir temp-dir-to-build-fpm-for-fiats-installation + curl -L -o temp-dir-to-build-fpm-for-fiats-installation/fpm.F90 https://github.com/fortran-lang/fpm/releases/download/current/fpm.F90 + gfortran -o $PREFIX/bin/fpm -Jtemp-dir-to-build-fpm-for-fiats-installation temp-dir-to-build-fpm-for-fiats-installation/fpm.F90 + rm -rf temp-dir-to-build-fpm-for-fiats-installation if command -v fpm > /dev/null ; then echo "fpm installed" else @@ -80,7 +80,7 @@ FPM_CC=${CC:-"gcc-14"} fpm test --profile release --flag "-fopenmp" echo "" -echo "____________________ Inference-Engine has been set up! _______________________" +echo "____________________ Fiats has been set up! _______________________" echo "" echo "Enter the command below to the see names of example use cases that you can run:" echo "" diff --git a/src/inference_engine/activation_m.f90 b/src/fiats/activation_m.f90 similarity index 85% rename from src/inference_engine/activation_m.f90 rename to src/fiats/activation_m.f90 index 966f6beea..9450a069c 100644 --- a/src/inference_engine/activation_m.f90 +++ b/src/fiats/activation_m.f90 @@ -20,10 +20,10 @@ module activation_m procedure, non_overridable :: function_name generic :: operator(==) => equals procedure, private :: equals - generic :: evaluate => default_real_evaluate , double_precision_evaluate - procedure, non_overridable :: default_real_evaluate , double_precision_evaluate - generic :: differentiate => default_real_differentiate, double_precision_differentiate - procedure, non_overridable :: default_real_differentiate, double_precision_differentiate + generic :: evaluate => default_real_evaluate , double_precision_evaluate + procedure, non_overridable, private :: default_real_evaluate , double_precision_evaluate + generic :: differentiate => default_real_differentiate, double_precision_differentiate + procedure, non_overridable, private :: default_real_differentiate, double_precision_differentiate end type interface activation_t diff --git a/src/inference_engine/activation_s.f90 b/src/fiats/activation_s.f90 similarity index 100% rename from src/inference_engine/activation_s.f90 rename to src/fiats/activation_s.f90 diff --git a/src/inference_engine/double_precision_file_m.f90 b/src/fiats/double_precision_file_m.f90 similarity index 100% rename from src/inference_engine/double_precision_file_m.f90 rename to src/fiats/double_precision_file_m.f90 diff --git a/src/inference_engine/double_precision_file_s.f90 b/src/fiats/double_precision_file_s.f90 similarity index 100% rename from src/inference_engine/double_precision_file_s.f90 rename to src/fiats/double_precision_file_s.f90 diff --git a/src/inference_engine/double_precision_string_m.f90 b/src/fiats/double_precision_string_m.f90 similarity index 100% rename from src/inference_engine/double_precision_string_m.f90 rename to src/fiats/double_precision_string_m.f90 diff --git a/src/inference_engine/double_precision_string_s.f90 b/src/fiats/double_precision_string_s.f90 similarity index 100% rename from src/inference_engine/double_precision_string_s.f90 rename to src/fiats/double_precision_string_s.f90 diff --git a/src/inference_engine/hyperparameters_m.f90 b/src/fiats/hyperparameters_m.f90 similarity index 100% rename from src/inference_engine/hyperparameters_m.f90 rename to src/fiats/hyperparameters_m.f90 diff --git a/src/inference_engine/hyperparameters_s.f90 b/src/fiats/hyperparameters_s.f90 similarity index 100% rename from src/inference_engine/hyperparameters_s.f90 rename to src/fiats/hyperparameters_s.f90 diff --git a/src/inference_engine/input_output_pair_m.f90 b/src/fiats/input_output_pair_m.f90 similarity index 100% rename from src/inference_engine/input_output_pair_m.f90 rename to src/fiats/input_output_pair_m.f90 diff --git a/src/inference_engine/input_output_pair_s.f90 b/src/fiats/input_output_pair_s.f90 similarity index 100% rename from src/inference_engine/input_output_pair_s.f90 rename to src/fiats/input_output_pair_s.f90 diff --git a/src/inference_engine/kind_parameters_m.f90 b/src/fiats/kind_parameters_m.f90 similarity index 100% rename from src/inference_engine/kind_parameters_m.f90 rename to src/fiats/kind_parameters_m.f90 diff --git a/src/inference_engine/layer_m.f90 b/src/fiats/layer_m.f90 similarity index 74% rename from src/inference_engine/layer_m.f90 rename to src/fiats/layer_m.f90 index cc418e8d6..e877d397a 100644 --- a/src/inference_engine/layer_m.f90 +++ b/src/fiats/layer_m.f90 @@ -4,7 +4,7 @@ module layer_m use double_precision_string_m, only : double_precision_string_t use kind_parameters_m, only : default_real, double_precision use julienne_string_m, only : string_t - use inference_engine_m_, only : inference_engine_t + use neural_network_m, only : neural_network_t use metadata_m, only : metadata_t use neuron_m, only : neuron_t use tensor_map_m, only : tensor_map_t @@ -19,20 +19,20 @@ module layer_m type(neuron_t(k)), private :: neuron !! linked list of this layer's neurons type(layer_t(k)), allocatable, private :: next !! next layer contains - generic :: inference_engine => default_real_inference_engine, double_precision_inference_engine - procedure, private :: default_real_inference_engine, double_precision_inference_engine - generic :: count_layers => default_real_count_layers, double_precision_count_layers - procedure, private :: default_real_count_layers, double_precision_count_layers - generic :: count_neurons => default_real_count_neurons, double_precision_count_neurons - procedure, private :: default_real_count_neurons, double_precision_count_neurons - generic :: count_inputs => default_real_count_inputs, double_precision_count_inputs - procedure, private :: default_real_count_inputs, double_precision_count_inputs + generic :: neural_network => default_real_neural_network , double_precision_neural_network + procedure, private :: default_real_neural_network , double_precision_neural_network + generic :: count_layers => default_real_count_layers , double_precision_count_layers + procedure, private :: default_real_count_layers , double_precision_count_layers + generic :: count_neurons => default_real_count_neurons , double_precision_count_neurons + procedure, private :: default_real_count_neurons , double_precision_count_neurons + generic :: count_inputs => default_real_count_inputs , double_precision_count_inputs + procedure, private :: default_real_count_inputs , double_precision_count_inputs generic :: neurons_per_layer => default_real_neurons_per_layer, double_precision_neurons_per_layer - procedure, private :: default_real_neurons_per_layer, double_precision_neurons_per_layer - generic :: next_allocated => default_real_next_allocated, double_precision_next_allocated - procedure, private :: default_real_next_allocated, double_precision_next_allocated - generic :: next_pointer => default_real_next_pointer, double_precision_next_pointer - procedure, private :: default_real_next_pointer, double_precision_next_pointer + procedure, private :: default_real_neurons_per_layer, double_precision_neurons_per_layer + generic :: next_allocated => default_real_next_allocated , double_precision_next_allocated + procedure, private :: default_real_next_allocated , double_precision_next_allocated + generic :: next_pointer => default_real_next_pointer , double_precision_next_pointer + procedure, private :: default_real_next_pointer , double_precision_next_pointer end type interface layer_t @@ -57,22 +57,22 @@ recursive module function double_precision_construct_layer(layer_lines, start) r interface - module function default_real_inference_engine(hidden_layers, metadata, output_layer, input_map, output_map) result(inference_engine_) + module function default_real_neural_network(hidden_layers, metadata, output_layer, input_map, output_map) result(neural_network_) implicit none class(layer_t), intent(in), target :: hidden_layers type(layer_t), intent(in), target :: output_layer type(string_t), intent(in) :: metadata(:) type(tensor_map_t), intent(in) :: input_map, output_map - type(inference_engine_t) inference_engine_ + type(neural_network_t) neural_network_ end function - module function double_precision_inference_engine(hidden_layers, metadata, output_layer, input_map, output_map) result(inference_engine_) + module function double_precision_neural_network(hidden_layers, metadata, output_layer, input_map, output_map) result(neural_network_) implicit none class(layer_t(double_precision)), intent(in), target :: hidden_layers type(layer_t(double_precision)), intent(in), target :: output_layer type(metadata_t), intent(in) :: metadata type(tensor_map_t(double_precision)), intent(in) :: input_map, output_map - type(inference_engine_t(double_precision)) inference_engine_ + type(neural_network_t(double_precision)) neural_network_ end function module function default_real_count_layers(layer) result(num_layers) diff --git a/src/inference_engine/layer_s.f90 b/src/fiats/layer_s.f90 similarity index 93% rename from src/inference_engine/layer_s.f90 rename to src/fiats/layer_s.f90 index 231514e2f..589740b36 100644 --- a/src/inference_engine/layer_s.f90 +++ b/src/fiats/layer_s.f90 @@ -68,7 +68,7 @@ end procedure - module procedure default_real_inference_engine + module procedure default_real_neural_network associate( & num_inputs => hidden_layers%count_inputs(), & @@ -77,7 +77,7 @@ num_hidden_layers => hidden_layers%count_layers(), & num_output_layers => output_layer%count_layers() & ) - call assert(num_output_layers==1, "inference_engine_s(default_real_inference_engine): 1 output layer", num_output_layers) + call assert(num_output_layers==1, "neural_network_s(default_real_neural_network): 1 output layer", num_output_layers) associate(nodes => [num_inputs, neurons_per_hidden_layer, num_outputs]) associate(n_max => maxval(nodes)) @@ -132,15 +132,15 @@ end do loop_over_output_neurons - inference_engine_ = inference_engine_t(metadata, weights, biases, nodes, input_map, output_map) + neural_network_ = neural_network_t(metadata, weights, biases, nodes, input_map, output_map) end block end associate end associate end associate - end procedure default_real_inference_engine + end procedure default_real_neural_network - module procedure double_precision_inference_engine + module procedure double_precision_neural_network associate( & num_inputs => hidden_layers%count_inputs(), & @@ -149,7 +149,7 @@ num_hidden_layers => hidden_layers%count_layers(), & num_output_layers => output_layer%count_layers() & ) - call assert(num_output_layers==1, "inference_engine_s(double_precision_inference_engine): 1 output layer", num_output_layers) + call assert(num_output_layers==1, "neural_network_s(double_precision_neural_network): 1 output layer", num_output_layers) associate(nodes => [num_inputs, neurons_per_hidden_layer, num_outputs]) associate(n_max => maxval(nodes)) @@ -204,13 +204,13 @@ end do loop_over_output_neurons - inference_engine_ = inference_engine_t(metadata, weights, biases, nodes, input_map, output_map) + neural_network_ = neural_network_t(metadata, weights, biases, nodes, input_map, output_map) end block end associate end associate end associate - end procedure double_precision_inference_engine + end procedure double_precision_neural_network module procedure default_real_count_layers diff --git a/src/inference_engine/metadata_m.f90 b/src/fiats/metadata_m.f90 similarity index 100% rename from src/inference_engine/metadata_m.f90 rename to src/fiats/metadata_m.f90 diff --git a/src/inference_engine/metadata_s.f90 b/src/fiats/metadata_s.f90 similarity index 100% rename from src/inference_engine/metadata_s.f90 rename to src/fiats/metadata_s.f90 diff --git a/src/inference_engine/mini_batch_m.f90 b/src/fiats/mini_batch_m.f90 similarity index 100% rename from src/inference_engine/mini_batch_m.f90 rename to src/fiats/mini_batch_m.f90 diff --git a/src/inference_engine/mini_batch_s.f90 b/src/fiats/mini_batch_s.f90 similarity index 100% rename from src/inference_engine/mini_batch_s.f90 rename to src/fiats/mini_batch_s.f90 diff --git a/src/inference_engine/network_configuration_m.f90 b/src/fiats/network_configuration_m.f90 similarity index 100% rename from src/inference_engine/network_configuration_m.f90 rename to src/fiats/network_configuration_m.f90 diff --git a/src/inference_engine/network_configuration_s.F90 b/src/fiats/network_configuration_s.F90 similarity index 100% rename from src/inference_engine/network_configuration_s.F90 rename to src/fiats/network_configuration_s.F90 diff --git a/src/inference_engine/inference_engine_m_.f90 b/src/fiats/neural_network_m.f90 similarity index 76% rename from src/inference_engine/inference_engine_m_.f90 rename to src/fiats/neural_network_m.f90 index 40121b4cf..50bdd7654 100644 --- a/src/inference_engine/inference_engine_m_.f90 +++ b/src/fiats/neural_network_m.f90 @@ -1,7 +1,7 @@ ! Copyright (c), The Regents of the University of California ! Terms of use are as specified in LICENSE.txt -module inference_engine_m_ - !! Define an abstraction that supports inference operationsn on a neural network +module neural_network_m + !! Define an abstraction that supports inference operations on a neural network use activation_m, only : activation_t use double_precision_file_m, only : double_precision_file_t use kind_parameters_m, only : default_real, double_precision @@ -13,13 +13,12 @@ module inference_engine_m_ implicit none private - public :: inference_engine_t + public :: neural_network_t public :: unmapped_network_t - public :: exchange_t public :: workspace_t - type inference_engine_t(k) - !! Encapsulate the minimal information needed to perform inference + type neural_network_t(k) + !! Encapsulate the information needed to perform inference integer, kind :: k = default_real type(tensor_map_t(k)), private :: input_map_, output_map_ type(metadata_t), private :: metadata_ @@ -39,7 +38,6 @@ module inference_engine_m_ generic :: assert_conformable_with => default_real_assert_conformable_with, double_precision_assert_conformable_with generic :: skip => default_real_skip, double_precision_skip generic :: activation_function_name => default_real_activation_name, double_precision_activation_name - generic :: to_exchange => default_real_to_exchange, double_precision_to_exchange generic :: learn => default_real_learn generic :: assert_consistency => default_real_consistency, double_precision_consistency procedure, private, non_overridable :: default_real_consistency, double_precision_consistency @@ -56,7 +54,6 @@ module inference_engine_m_ procedure, private, non_overridable :: default_real_assert_conformable_with, double_precision_assert_conformable_with procedure, private, non_overridable :: default_real_skip, double_precision_skip procedure, private, non_overridable :: default_real_activation_name, double_precision_activation_name - procedure, private, non_overridable :: default_real_to_exchange, double_precision_to_exchange end type type workspace_t(k) @@ -73,9 +70,9 @@ module inference_engine_m_ interface workspace_t - pure module function default_real_workspace(inference_engine) result(workspace) + pure module function default_real_workspace(neural_network) result(workspace) implicit none - type(inference_engine_t), intent(in) :: inference_engine + type(neural_network_t), intent(in) :: neural_network type(workspace_t) workspace end function @@ -83,10 +80,10 @@ pure module function default_real_workspace(inference_engine) result(workspace) interface - module subroutine default_real_allocate(self, inference_engine) + module subroutine default_real_allocate(self, neural_network) implicit none class(workspace_t), intent(inout) :: self - type(inference_engine_t), intent(in) :: inference_engine + type(neural_network_t), intent(in) :: neural_network end subroutine pure module function default_real_allocated(self) result(all_allocated) @@ -97,90 +94,62 @@ pure module function default_real_allocated(self) result(all_allocated) end interface - type exchange_t(k) - integer, kind :: k = default_real - type(tensor_map_t(k)) input_map_, output_map_ - type(metadata_t) metadata_ - real(k), allocatable :: weights_(:,:,:), biases_(:,:) - integer, allocatable :: nodes_(:) - type(activation_t) activation_ - end type - - interface - - module function default_real_to_exchange(self) result(exchange) - implicit none - class(inference_engine_t), intent(in) :: self - type(exchange_t) exchange - end function - - module function double_precision_to_exchange(self) result(exchange) - implicit none - class(inference_engine_t(double_precision)), intent(in) :: self - type(exchange_t(double_precision)) exchange - end function - - end interface - - interface inference_engine_t + interface neural_network_t module function default_real_construct_from_components(metadata, weights, biases, nodes, input_map, output_map) & - result(inference_engine) + result(neural_network) implicit none type(string_t), intent(in) :: metadata(:) real, intent(in) :: weights(:,:,:), biases(:,:) integer, intent(in) :: nodes(0:) type(tensor_map_t), intent(in), optional :: input_map, output_map - type(inference_engine_t) inference_engine + type(neural_network_t) neural_network end function module function double_precision_construct_from_components(metadata, weights, biases, nodes, input_map, output_map) & - result(inference_engine) + result(neural_network) implicit none type(metadata_t), intent(in) :: metadata double precision, intent(in) :: weights(:,:,:), biases(:,:) integer, intent(in) :: nodes(0:) type(tensor_map_t(double_precision)), intent(in), optional :: input_map, output_map - type(inference_engine_t(double_precision)) inference_engine + type(neural_network_t(double_precision)) neural_network end function - impure elemental module function default_real_from_json(file_) result(inference_engine) + impure elemental module function default_real_from_json(file_) result(neural_network) implicit none type(file_t), intent(in) :: file_ - type(inference_engine_t) inference_engine + type(neural_network_t) neural_network end function - impure elemental module function double_precision_from_json(file) result(inference_engine) + impure elemental module function double_precision_from_json(file) result(neural_network) implicit none type(double_precision_file_t), intent(in) :: file - type(inference_engine_t(double_precision)) inference_engine + type(neural_network_t(double_precision)) neural_network end function end interface - - - - interface ! inference_engine_t type-bound procedures + interface ! neural_network_t type-bound procedures elemental module function default_real_approximately_equal(lhs, rhs) result(lhs_eq_rhs) !! The result is true if lhs and rhs are the same to within a tolerance implicit none - class(inference_engine_t), intent(in) :: lhs, rhs + class(neural_network_t), intent(in) :: lhs, rhs logical lhs_eq_rhs end function elemental module function double_precision_approximately_equal(lhs, rhs) result(lhs_eq_rhs) !! The result is true if lhs and rhs are the same to within a tolerance implicit none - class(inference_engine_t(double_precision)), intent(in) :: lhs, rhs + class(neural_network_t(double_precision)), intent(in) :: lhs, rhs logical lhs_eq_rhs end function elemental module function default_real_map_to_input_range(self, tensor) result(normalized_tensor) !! The result contains the input tensor values normalized to fall on the range used during training implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self type(tensor_t), intent(in) :: tensor type(tensor_t) normalized_tensor end function @@ -188,7 +157,7 @@ elemental module function default_real_map_to_input_range(self, tensor) result(n elemental module function double_precision_map_to_input_range(self, tensor) result(normalized_tensor) !! The result contains the input tensor values normalized to fall on the range used during training implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self type(tensor_t(double_precision)), intent(in) :: tensor type(tensor_t(double_precision)) normalized_tensor end function @@ -196,7 +165,7 @@ elemental module function double_precision_map_to_input_range(self, tensor) resu elemental module function default_real_map_from_output_range(self, normalized_tensor) result(tensor) !! The result contains the output tensor values unmapped via the inverse of the mapping used in training implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self type(tensor_t), intent(in) :: normalized_tensor type(tensor_t) tensor end function @@ -204,124 +173,124 @@ elemental module function default_real_map_from_output_range(self, normalized_te elemental module function double_precision_map_from_output_range(self, normalized_tensor) result(tensor) !! The result contains the output tensor values unmapped via the inverse of the mapping used in training implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self type(tensor_t(double_precision)), intent(in) :: normalized_tensor type(tensor_t(double_precision)) tensor end function impure elemental module function default_real_to_json(self) result(json_file) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self type(file_t) json_file end function impure elemental module function double_precision_to_json(self) result(json_file) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self type(file_t) json_file end function - elemental module subroutine default_real_assert_conformable_with(self, inference_engine) + elemental module subroutine default_real_assert_conformable_with(self, neural_network) implicit none - class(inference_engine_t), intent(in) :: self - type(inference_engine_t), intent(in) :: inference_engine + class(neural_network_t), intent(in) :: self + type(neural_network_t), intent(in) :: neural_network end subroutine - elemental module subroutine double_precision_assert_conformable_with(self, inference_engine) + elemental module subroutine double_precision_assert_conformable_with(self, neural_network) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self - type(inference_engine_t(double_precision)), intent(in) :: inference_engine + class(neural_network_t(double_precision)), intent(in) :: self + type(neural_network_t(double_precision)), intent(in) :: neural_network end subroutine elemental module function default_real_infer(self, inputs) result(outputs) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self type(tensor_t), intent(in) :: inputs type(tensor_t) outputs end function elemental module function double_precision_infer(self, inputs) result(outputs) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self type(tensor_t(double_precision)), intent(in) :: inputs type(tensor_t(double_precision)) outputs end function elemental module function default_real_num_outputs(self) result(output_count) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self integer output_count end function elemental module function double_precision_num_outputs(self) result(output_count) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self integer output_count end function elemental module function default_real_num_hidden_layers(self) result(hidden_layer_count) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self integer hidden_layer_count end function elemental module function double_precision_num_hidden_layers(self) result(hidden_layer_count) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self integer hidden_layer_count end function elemental module function default_real_num_inputs(self) result(input_count) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self integer input_count end function elemental module function double_precision_num_inputs(self) result(input_count) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self integer input_count end function pure module function default_real_nodes_per_layer(self) result(node_count) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self integer, allocatable :: node_count(:) end function pure module function double_precision_nodes_per_layer(self) result(node_count) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self integer, allocatable :: node_count(:) end function elemental module function default_real_activation_name(self) result(activation_name) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self type(string_t) activation_name end function elemental module function double_precision_activation_name(self) result(activation_name) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self type(string_t) activation_name end function pure module function default_real_skip(self) result(use_skip_connections) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self logical use_skip_connections end function pure module function double_precision_skip(self) result(use_skip_connections) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self logical use_skip_connections end function pure module subroutine default_real_learn(self, mini_batches_arr, cost, adam, learning_rate, workspace) implicit none - class(inference_engine_t), intent(inout) :: self + class(neural_network_t), intent(inout) :: self type(mini_batch_t), intent(in) :: mini_batches_arr(:) real, intent(out), allocatable, optional :: cost(:) logical, intent(in) :: adam @@ -331,12 +300,12 @@ pure module subroutine default_real_learn(self, mini_batches_arr, cost, adam, le pure module subroutine default_real_consistency(self) implicit none - class(inference_engine_t), intent(in) :: self + class(neural_network_t), intent(in) :: self end subroutine pure module subroutine double_precision_consistency(self) implicit none - class(inference_engine_t(double_precision)), intent(in) :: self + class(neural_network_t(double_precision)), intent(in) :: self end subroutine end interface @@ -344,7 +313,7 @@ pure module subroutine double_precision_consistency(self) type unmapped_network_t(k) integer, kind :: k = default_real private - type(inference_engine_t(k)) inference_engine_ + type(neural_network_t(k)) neural_network_ contains generic :: infer => default_real_infer_unmapped, double_precision_infer_unmapped procedure, private, non_overridable :: default_real_infer_unmapped, double_precision_infer_unmapped @@ -378,4 +347,4 @@ elemental module function double_precision_infer_unmapped(self, inputs) result(o end interface -end module inference_engine_m_ +end module neural_network_m diff --git a/src/inference_engine/inference_engine_s.F90 b/src/fiats/neural_network_s.F90 similarity index 87% rename from src/inference_engine/inference_engine_s.F90 rename to src/fiats/neural_network_s.F90 index ac31642f6..6f3334c80 100644 --- a/src/inference_engine/inference_engine_s.F90 +++ b/src/fiats/neural_network_s.F90 @@ -1,6 +1,6 @@ ! Copyright (c), The Regents of the University of California ! Terms of use are as specified in LICENSE.txt -submodule(inference_engine_m_) inference_engine_s +submodule(neural_network_m) neural_network_s use assert_m, only : assert, intrinsic_array_t use double_precision_string_m, only : double_precision_string_t use kind_parameters_m, only : double_precision @@ -8,7 +8,7 @@ use neuron_m, only : neuron_t implicit none - character(len=*), parameter :: acceptable_engine_tag = "0.13.0" ! git tag capable of reading the current json file format + character(len=*), parameter :: minimum_acceptable_tag = "0.15.0" ! git tag capable of reading the current json file format integer, parameter :: input_layer = 0 contains @@ -29,30 +29,6 @@ tensor = self%output_map_%map_from_training_range(normalized_tensor) end procedure - module procedure default_real_to_exchange - exchange%input_map_ = self%input_map_ - exchange%output_map_ = self%output_map_ - associate(strings => self%metadata_%strings()) - exchange%metadata_ = metadata_t(strings(1),strings(2),strings(3),strings(4),strings(5)) - end associate - exchange%weights_ = self%weights_ - exchange%biases_ = self%biases_ - exchange%nodes_ = self%nodes_ - exchange%activation_ = self%activation_ - end procedure - - module procedure double_precision_to_exchange - exchange%input_map_ = self%input_map_ - exchange%output_map_ = self%output_map_ - associate(strings => self%metadata_%strings()) - exchange%metadata_ = metadata_t(strings(1),strings(2),strings(3),strings(4),strings(5)) - end associate - exchange%weights_ = self%weights_ - exchange%biases_ = self%biases_ - exchange%nodes_ = self%nodes_ - exchange%activation_ = self%activation_ - end procedure - module procedure default_real_infer real, allocatable :: a(:,:) @@ -152,17 +128,17 @@ associate( & all_allocated=>[allocated(self%weights_),allocated(self%biases_),allocated(self%nodes_)]& ) - call assert(all(all_allocated),"inference_engine_s(default_real_consistency): fully_allocated", & + call assert(all(all_allocated),"neural_network_s(default_real_consistency): fully_allocated", & intrinsic_array_t(all_allocated)) end associate associate(max_width=>maxval(self%nodes_), component_dims=>[size(self%biases_,1), size(self%weights_,1), size(self%weights_,2)]) - call assert(all(component_dims == max_width), "inference_engine_s(default_real_consistency): conformable arrays", & + call assert(all(component_dims == max_width), "neural_network_s(default_real_consistency): conformable arrays", & intrinsic_array_t([max_width,component_dims])) end associate associate(input_subscript => lbound(self%nodes_,1)) - call assert(input_subscript == input_layer, "inference_engine_s(default_real_consistency): n base subsscript", & + call assert(input_subscript == input_layer, "neural_network_s(default_real_consistency): n base subsscript", & input_subscript) end associate @@ -173,17 +149,17 @@ associate( & all_allocated=>[allocated(self%weights_),allocated(self%biases_),allocated(self%nodes_)]& ) - call assert(all(all_allocated),"inference_engine_s(default_real_consistency): fully_allocated", & + call assert(all(all_allocated),"neural_network_s(default_real_consistency): fully_allocated", & intrinsic_array_t(all_allocated)) end associate associate(max_width=>maxval(self%nodes_), component_dims=>[size(self%biases_,1), size(self%weights_,1), size(self%weights_,2)]) - call assert(all(component_dims == max_width), "inference_engine_s(default_real_consistency): conformable arrays", & + call assert(all(component_dims == max_width), "neural_network_s(default_real_consistency): conformable arrays", & intrinsic_array_t([max_width,component_dims])) end associate associate(input_subscript => lbound(self%nodes_,1)) - call assert(input_subscript == input_layer, "inference_engine_s(default_real_consistency): n base subsscript", & + call assert(input_subscript == input_layer, "neural_network_s(default_real_consistency): n base subsscript", & input_subscript) end associate @@ -191,77 +167,77 @@ module procedure default_real_construct_from_components - inference_engine%metadata_ = metadata_t(metadata(1),metadata(2),metadata(3),metadata(4),metadata(5)) - inference_engine%weights_ = weights - inference_engine%biases_ = biases - inference_engine%nodes_ = nodes + neural_network%metadata_ = metadata_t(metadata(1),metadata(2),metadata(3),metadata(4),metadata(5)) + neural_network%weights_ = weights + neural_network%biases_ = biases + neural_network%nodes_ = nodes block integer i if (present(input_map)) then - inference_engine%input_map_ = input_map + neural_network%input_map_ = input_map else associate(num_inputs => nodes(lbound(nodes,1))) associate(default_minima => [(0., i=1,num_inputs)], default_maxima => [(1., i=1,num_inputs)]) - inference_engine%input_map_ = tensor_map_t("inputs", default_minima, default_maxima) + neural_network%input_map_ = tensor_map_t("inputs", default_minima, default_maxima) end associate end associate end if if (present(output_map)) then - inference_engine%output_map_ = output_map + neural_network%output_map_ = output_map else associate(num_outputs => nodes(ubound(nodes,1))) associate(default_minima => [(0., i=1,num_outputs)], default_maxima => [(1., i=1,num_outputs)]) - inference_engine%output_map_ = tensor_map_t("outputs", default_minima, default_maxima) + neural_network%output_map_ = tensor_map_t("outputs", default_minima, default_maxima) end associate end associate end if end block - inference_engine%activation_ = activation_t(metadata(4)%string()) + neural_network%activation_ = activation_t(metadata(4)%string()) - call inference_engine%assert_consistency() + call neural_network%assert_consistency() end procedure default_real_construct_from_components module procedure double_precision_construct_from_components - inference_engine%metadata_ = metadata - inference_engine%weights_ = weights - inference_engine%biases_ = biases - inference_engine%nodes_ = nodes + neural_network%metadata_ = metadata + neural_network%weights_ = weights + neural_network%biases_ = biases + neural_network%nodes_ = nodes block integer i if (present(input_map)) then - inference_engine%input_map_ = input_map + neural_network%input_map_ = input_map else associate(num_inputs => nodes(lbound(nodes,1))) associate(default_intercept => [(0D0, i=1,num_inputs)], default_slope => [(1D0, i=1,num_inputs)]) - inference_engine%input_map_ = tensor_map_t("inputs", default_intercept, default_slope) + neural_network%input_map_ = tensor_map_t("inputs", default_intercept, default_slope) end associate end associate end if if (present(output_map)) then - inference_engine%output_map_ = output_map + neural_network%output_map_ = output_map else associate(num_outputs => nodes(ubound(nodes,1))) associate(default_intercept => [(0D0, i=1,num_outputs)], default_slope => [(1D0, i=1,num_outputs)]) - inference_engine%output_map_ = tensor_map_t("outputs", default_intercept, default_slope) + neural_network%output_map_ = tensor_map_t("outputs", default_intercept, default_slope) end associate end associate end if end block associate(function_name => metadata%activation_name()) - inference_engine%activation_ = activation_t(function_name%string()) + neural_network%activation_ = activation_t(function_name%string()) end associate - call inference_engine%assert_consistency() + call neural_network%assert_consistency() end procedure double_precision_construct_from_components @@ -303,7 +279,7 @@ associate( json_lines => & brace + & ! { - file_version_lines + & ! "acceptable_engine_tag": ... + file_version_lines + & ! "minimum_acceptable_tag": ... metadata_lines + & ! "metadata": ... tensor_map_lines + & ! "inputs_tensor_map": ... tensor_map_lines + & ! "outputs_tensor_map": ... @@ -319,7 +295,7 @@ ) allocate(lines(json_lines)) lines(brace) = string_t('{') - lines(brace+1:brace+file_version_lines)= string_t(' "acceptable_engine_tag": "')//acceptable_engine_tag//'",' + lines(brace+1:brace+file_version_lines)= string_t(' "minimum_acceptable_tag": "')//minimum_acceptable_tag//'",' associate(meta_start => brace + file_version_lines + 1) associate(meta_end => meta_start + metadata_lines - 1) lines(meta_start:meta_end) = self%metadata_%to_json() @@ -373,7 +349,7 @@ lines(line) = string_t(' ]') line = line + 1 lines(line) = string_t('}') - call assert(line == json_lines, "inference_engine_t%to_json: all lines defined", intrinsic_array_t([json_lines, line])) + call assert(line == json_lines, "neural_network_t%to_json: all lines defined", intrinsic_array_t([json_lines, line])) end associate json_file = file_t(lines) end block @@ -419,7 +395,7 @@ associate( json_lines => & brace + & ! { - file_version_lines + & ! "acceptable_engine_tag": ... + file_version_lines + & ! "minimum_acceptable_tag": ... metadata_lines + & ! "metadata": ... tensor_map_lines + & ! "inputs_tensor_map": ... tensor_map_lines + & ! "outputs_tensor_map": ... @@ -435,7 +411,7 @@ ) allocate(lines(json_lines)) lines(brace) = string_t('{') - lines(brace+1:brace+file_version_lines)= string_t(' "acceptable_engine_tag": "')//acceptable_engine_tag//'",' + lines(brace+1:brace+file_version_lines)= string_t(' "minimum_acceptable_tag": "')//minimum_acceptable_tag//'",' associate(meta_start => brace + file_version_lines + 1) associate(meta_end => meta_start + metadata_lines - 1) lines(meta_start:meta_end) = self%metadata_%to_json() @@ -489,7 +465,7 @@ lines(line) = string_t(' ]') line = line + 1 lines(line) = string_t('}') - call assert(line == json_lines, "inference_engine_t%to_json: all lines defined", intrinsic_array_t([json_lines, line])) + call assert(line == json_lines, "neural_network_t%to_json: all lines defined", intrinsic_array_t([json_lines, line])) end associate json_file = file_t(lines) end block @@ -506,17 +482,17 @@ type(layer_t) hidden_layers, output_layer lines = file_%lines() - call assert(adjustl(lines(1)%string())=="{", "inference_engine_s(default_real_from_json): expected outermost object '{'") + call assert(adjustl(lines(1)%string())=="{", "neural_network_s(default_real_from_json): expected outermost object '{'") check_git_tag: & block character(len=:), allocatable :: tag - tag = lines(2)%get_json_value("acceptable_engine_tag", mold="") + tag = lines(2)%get_json_value("minimum_acceptable_tag", mold="") call assert( & - tag == acceptable_engine_tag & - ,"inference_engine_s(default_real_from_json): acceptable_engine_tag" & - ,tag //"(expected " //acceptable_engine_tag // ")" & + tag == minimum_acceptable_tag & + ,"neural_network_s(default_real_from_json): minimum_acceptable_tag" & + ,tag //"(expected " //minimum_acceptable_tag // ")" & ) end block check_git_tag @@ -587,15 +563,15 @@ associate(proto_meta => metadata_t(string_t(""),string_t(""),string_t(""),string_t(""),string_t(""))) associate(metadata => metadata_t(lines(l : l + size(proto_meta%to_json()) - 1))) associate(metadata_strings => metadata%strings()) - inference_engine = hidden_layers%inference_engine(metadata_strings, output_layer, input_map, output_map) + neural_network = hidden_layers%neural_network(metadata_strings, output_layer, input_map, output_map) associate(function_name => metadata%activation_name()) - inference_engine%activation_ = activation_t(function_name%string()) + neural_network%activation_ = activation_t(function_name%string()) end associate end associate end associate end associate read_metadata - call inference_engine%assert_consistency() + call neural_network%assert_consistency() end procedure default_real_from_json @@ -608,17 +584,17 @@ type(layer_t(double_precision)) hidden_layers, output_layer lines = file%double_precision_lines() - call assert(adjustl(lines(1)%string())=="{", "inference_engine_s(double_precision_from_json): expected outermost object '{'") + call assert(adjustl(lines(1)%string())=="{", "neural_network_s(double_precision_from_json): expected outermost object '{'") check_git_tag: & block character(len=:), allocatable :: tag - tag = lines(2)%get_json_value("acceptable_engine_tag", mold="") + tag = lines(2)%get_json_value("minimum_acceptable_tag", mold="") call assert( & - tag == acceptable_engine_tag & - ,"inference_engine_s(double_precision_from_json): acceptable_engine_tag" & - ,tag //"(expected " //acceptable_engine_tag // ")" & + tag == minimum_acceptable_tag & + ,"neural_network_s(double_precision_from_json): minimum_acceptable_tag" & + ,tag //"(expected " //minimum_acceptable_tag // ")" & ) end block check_git_tag @@ -688,14 +664,14 @@ read_metadata: & associate(proto_meta => metadata_t(string_t(""),string_t(""),string_t(""),string_t(""),string_t(""))) associate(metadata => metadata_t(lines(l : l + size(proto_meta%to_json()) - 1))) - inference_engine = hidden_layers%inference_engine(metadata, output_layer, input_map, output_map) + neural_network = hidden_layers%neural_network(metadata, output_layer, input_map, output_map) associate(function_name => metadata%activation_name()) - inference_engine%activation_ = activation_t(function_name%string()) + neural_network%activation_ = activation_t(function_name%string()) end associate end associate end associate read_metadata - call inference_engine%assert_consistency() + call neural_network%assert_consistency() end procedure double_precision_from_json @@ -704,14 +680,14 @@ call self%assert_consistency() associate(equal_shapes => [ & - shape(self%weights_) == shape(inference_engine%weights_), & - shape(self%biases_) == shape(inference_engine%biases_), & - shape(self%nodes_) == shape(inference_engine%nodes_) & + shape(self%weights_) == shape(neural_network%weights_), & + shape(self%biases_) == shape(neural_network%biases_), & + shape(self%nodes_) == shape(neural_network%nodes_) & ]) call assert(all(equal_shapes), "assert_conformable_with: all(equal_shapes)", intrinsic_array_t(equal_shapes)) end associate - call assert(self%activation_ == inference_engine%activation_, "assert_conformable_with: activation_") + call assert(self%activation_ == neural_network%activation_, "assert_conformable_with: activation_") end procedure @@ -720,14 +696,14 @@ call self%assert_consistency() associate(equal_shapes => [ & - shape(self%weights_) == shape(inference_engine%weights_), & - shape(self%biases_) == shape(inference_engine%biases_), & - shape(self%nodes_) == shape(inference_engine%nodes_) & + shape(self%weights_) == shape(neural_network%weights_), & + shape(self%biases_) == shape(neural_network%biases_), & + shape(self%nodes_) == shape(neural_network%nodes_) & ]) call assert(all(equal_shapes), "assert_conformable_with: all(equal_shapes)", intrinsic_array_t(equal_shapes)) end associate - call assert(self%activation_ == inference_engine%activation_, "assert_conformable_with: activation_") + call assert(self%activation_ == neural_network%activation_, "assert_conformable_with: activation_") end procedure @@ -888,7 +864,7 @@ type(tensor_t), allocatable :: inputs(:), expected_outputs(:) call self%assert_consistency() - call assert(workspace%fully_allocated(), "inference_engine_s(default_real_learn): workspace%fully_allocated()") + call assert(workspace%fully_allocated(), "neural_network_s(default_real_learn): workspace%fully_allocated()") associate(output_layer => ubound(self%nodes_,1)) @@ -1072,4 +1048,4 @@ end associate end procedure default_real_learn -end submodule inference_engine_s +end submodule neural_network_s diff --git a/src/inference_engine/neuron_m.f90 b/src/fiats/neuron_m.f90 similarity index 100% rename from src/inference_engine/neuron_m.f90 rename to src/fiats/neuron_m.f90 diff --git a/src/inference_engine/neuron_s.f90 b/src/fiats/neuron_s.f90 similarity index 100% rename from src/inference_engine/neuron_s.f90 rename to src/fiats/neuron_s.f90 diff --git a/src/inference_engine/tensor_m.f90 b/src/fiats/tensor_m.f90 similarity index 100% rename from src/inference_engine/tensor_m.f90 rename to src/fiats/tensor_m.f90 diff --git a/src/inference_engine/tensor_map_m.f90 b/src/fiats/tensor_map_m.f90 similarity index 100% rename from src/inference_engine/tensor_map_m.f90 rename to src/fiats/tensor_map_m.f90 diff --git a/src/inference_engine/tensor_map_s.f90 b/src/fiats/tensor_map_s.f90 similarity index 100% rename from src/inference_engine/tensor_map_s.f90 rename to src/fiats/tensor_map_s.f90 diff --git a/src/inference_engine/tensor_s.f90 b/src/fiats/tensor_s.f90 similarity index 100% rename from src/inference_engine/tensor_s.f90 rename to src/fiats/tensor_s.f90 diff --git a/src/inference_engine/trainable_network_m.f90 b/src/fiats/trainable_network_m.f90 similarity index 88% rename from src/inference_engine/trainable_network_m.f90 rename to src/fiats/trainable_network_m.f90 index 86380eb0f..0cb2cfc50 100644 --- a/src/inference_engine/trainable_network_m.f90 +++ b/src/fiats/trainable_network_m.f90 @@ -1,5 +1,5 @@ module trainable_network_m - use inference_engine_m_, only : inference_engine_t, workspace_t + use neural_network_m, only : neural_network_t, workspace_t use input_output_pair_m, only : input_output_pair_t use julienne_m, only : string_t use kind_parameters_m, only : default_real @@ -11,7 +11,7 @@ module trainable_network_m private public :: trainable_network_t - type, extends(inference_engine_t) :: trainable_network_t(k) + type, extends(neural_network_t) :: trainable_network_t(k) integer, kind :: k = default_real private type(workspace_t), private :: workspace_ @@ -24,9 +24,9 @@ module trainable_network_m interface trainable_network_t - pure module function default_real_network(inference_engine) result(trainable_network) + pure module function default_real_network(neural_network) result(trainable_network) implicit none - type(inference_engine_t), intent(in) :: inference_engine + type(neural_network_t), intent(in) :: neural_network type(trainable_network_t) trainable_network end function diff --git a/src/inference_engine/trainable_network_s.f90 b/src/fiats/trainable_network_s.f90 similarity index 88% rename from src/inference_engine/trainable_network_s.f90 rename to src/fiats/trainable_network_s.f90 index a4e3caa6c..12c025661 100644 --- a/src/inference_engine/trainable_network_s.f90 +++ b/src/fiats/trainable_network_s.f90 @@ -4,8 +4,8 @@ contains module procedure default_real_network - trainable_network%inference_engine_t = inference_engine - trainable_network%workspace_ = workspace_t(inference_engine) + trainable_network%neural_network_t = neural_network + trainable_network%workspace_ = workspace_t(neural_network) end procedure module procedure default_real_train @@ -42,7 +42,7 @@ b => perturbation_magnitude*(b_harvest-0.5)/0.5 & ) trainable_network = trainable_network_t( & - inference_engine_t(nodes=n, weights=w, biases=b, metadata=metadata, input_map=input_map, output_map=output_map) & + neural_network_t(nodes=n, weights=w, biases=b, metadata=metadata, input_map=input_map, output_map=output_map) & ) end associate end associate diff --git a/src/inference_engine/training_configuration_m.f90 b/src/fiats/training_configuration_m.f90 similarity index 100% rename from src/inference_engine/training_configuration_m.f90 rename to src/fiats/training_configuration_m.f90 diff --git a/src/inference_engine/training_configuration_s.F90 b/src/fiats/training_configuration_s.F90 similarity index 100% rename from src/inference_engine/training_configuration_s.F90 rename to src/fiats/training_configuration_s.F90 diff --git a/src/inference_engine/ubounds_m.f90 b/src/fiats/ubounds_m.f90 similarity index 100% rename from src/inference_engine/ubounds_m.f90 rename to src/fiats/ubounds_m.f90 diff --git a/src/inference_engine/unmapped_network_s.f90 b/src/fiats/unmapped_network_s.f90 similarity index 66% rename from src/inference_engine/unmapped_network_s.f90 rename to src/fiats/unmapped_network_s.f90 index 9ced289e5..4a2cf6e43 100644 --- a/src/inference_engine/unmapped_network_s.f90 +++ b/src/fiats/unmapped_network_s.f90 @@ -1,6 +1,6 @@ ! Copyright (c), The Regents of the University of California ! Terms of use are as specified in LICENSE.txt -submodule(inference_engine_m_) unmapped_network_s +submodule(neural_network_m) unmapped_network_s implicit none integer, parameter :: input_layer = 0 @@ -8,7 +8,7 @@ contains module procedure double_precision_unmapped_from_json - unmapped_network%inference_engine_ = double_precision_from_json(file) + unmapped_network%neural_network_ = double_precision_from_json(file) end procedure module procedure default_real_infer_unmapped @@ -16,15 +16,15 @@ real, allocatable :: a(:,:) integer l - associate(inference_engine => self%inference_engine_) + associate(neural_network => self%neural_network_) - call inference_engine%assert_consistency() + call neural_network%assert_consistency() associate( & - w => inference_engine%weights_ & - ,b => inference_engine%biases_ & - ,n => inference_engine%nodes_ & - ,output_layer => ubound(inference_engine%nodes_,1) & + w => neural_network%weights_ & + ,b => neural_network%biases_ & + ,n => neural_network%nodes_ & + ,output_layer => ubound(neural_network%nodes_,1) & ) allocate(a(maxval(n), input_layer:output_layer)) @@ -34,7 +34,7 @@ do l = input_layer+1, output_layer associate(z => matmul(w(1:n(l),1:n(l-1),l), a(1:n(l-1),l-1)) + b(1:n(l),l)) if (l .lt. output_layer) then - a(1:n(l),l) = inference_engine%activation_%evaluate(z) + a(1:n(l),l) = neural_network%activation_%evaluate(z) else a(1:n(l),l) = z(1:n(l)) end if @@ -53,15 +53,15 @@ double precision, allocatable :: a(:,:) integer l - associate(inference_engine => self%inference_engine_) + associate(neural_network => self%neural_network_) - call inference_engine%assert_consistency() + call neural_network%assert_consistency() associate( & - w => inference_engine%weights_ & - ,b => inference_engine%biases_ & - ,n => inference_engine%nodes_ & - ,output_layer => ubound(inference_engine%nodes_,1) & + w => neural_network%weights_ & + ,b => neural_network%biases_ & + ,n => neural_network%nodes_ & + ,output_layer => ubound(neural_network%nodes_,1) & ) allocate(a(maxval(n), input_layer:output_layer)) @@ -72,7 +72,7 @@ do l = input_layer+1, output_layer associate(z => matmul(w(1:n(l),1:n(l-1),l), a(1:n(l-1),l-1)) + b(1:n(l),l)) if (l .lt. output_layer) then - a(1:n(l),l) = inference_engine%activation_%evaluate(z) + a(1:n(l),l) = neural_network%activation_%evaluate(z) else a(1:n(l),l) = z(1:n(l)) end if diff --git a/src/inference_engine/workspace_s.f90 b/src/fiats/workspace_s.f90 similarity index 52% rename from src/inference_engine/workspace_s.f90 rename to src/fiats/workspace_s.f90 index f3d80139c..f3e1209d6 100644 --- a/src/inference_engine/workspace_s.f90 +++ b/src/fiats/workspace_s.f90 @@ -1,4 +1,4 @@ -submodule(inference_engine_m_) workspace_s +submodule(neural_network_m) workspace_s use assert_m, only : assert implicit none @@ -8,26 +8,26 @@ module procedure default_real_workspace - allocate(workspace%dcdw, mold=inference_engine%weights_) ! Gradient of cost function with respect to weights - allocate(workspace%vdw , mold=inference_engine%weights_) - allocate(workspace%sdw , mold=inference_engine%weights_) - allocate(workspace%vdwc, mold=inference_engine%weights_) - allocate(workspace%sdwc, mold=inference_engine%weights_) + allocate(workspace%dcdw, mold=neural_network%weights_) ! Gradient of cost function with respect to weights + allocate(workspace%vdw , mold=neural_network%weights_) + allocate(workspace%sdw , mold=neural_network%weights_) + allocate(workspace%vdwc, mold=neural_network%weights_) + allocate(workspace%sdwc, mold=neural_network%weights_) - allocate(workspace%dcdb, mold=inference_engine%biases_ ) ! Gradient of cost function with respect with biases - allocate(workspace%vdb , mold=inference_engine%biases_ ) - allocate(workspace%sdb , mold=inference_engine%biases_ ) - allocate(workspace%vdbc, mold=inference_engine%biases_ ) - allocate(workspace%sdbc, mold=inference_engine%biases_ ) + allocate(workspace%dcdb, mold=neural_network%biases_ ) ! Gradient of cost function with respect with biases + allocate(workspace%vdb , mold=neural_network%biases_ ) + allocate(workspace%sdb , mold=neural_network%biases_ ) + allocate(workspace%vdbc, mold=neural_network%biases_ ) + allocate(workspace%sdbc, mold=neural_network%biases_ ) ! TODO: #if ! (F2023_LOCALITY || F2018_LOCALITY) ! then don't allocate a, z, and delta - allocate(workspace%z , mold=inference_engine%biases_) - allocate(workspace%delta, mold=inference_engine%biases_) + allocate(workspace%z , mold=neural_network%biases_) + allocate(workspace%delta, mold=neural_network%biases_) - associate(output_layer => ubound(inference_engine%nodes_,1)) - allocate(workspace%a(maxval(inference_engine%nodes_), input_layer:output_layer)) ! Activations + associate(output_layer => ubound(neural_network%nodes_,1)) + allocate(workspace%a(maxval(neural_network%nodes_), input_layer:output_layer)) ! Activations end associate call assert(workspace%fully_allocated(), "workspace_s(defalt_real_workspace): workspace allocated") @@ -49,26 +49,26 @@ module procedure default_real_allocate - call allocate_if_necessary(self%dcdw, mold=inference_engine%weights_) ! Gradient of cost function with respect to weights - call allocate_if_necessary(self%vdw , mold=inference_engine%weights_) - call allocate_if_necessary(self%sdw , mold=inference_engine%weights_) - call allocate_if_necessary(self%vdwc, mold=inference_engine%weights_) - call allocate_if_necessary(self%sdwc, mold=inference_engine%weights_) + call allocate_if_necessary(self%dcdw, mold=neural_network%weights_) ! Gradient of cost function with respect to weights + call allocate_if_necessary(self%vdw , mold=neural_network%weights_) + call allocate_if_necessary(self%sdw , mold=neural_network%weights_) + call allocate_if_necessary(self%vdwc, mold=neural_network%weights_) + call allocate_if_necessary(self%sdwc, mold=neural_network%weights_) - call allocate_if_necessary(self%dcdb , mold=inference_engine%biases_) ! Gradient of cost function with respect with biases - call allocate_if_necessary(self%vdb , mold=inference_engine%biases_) - call allocate_if_necessary(self%sdb , mold=inference_engine%biases_) - call allocate_if_necessary(self%vdbc , mold=inference_engine%biases_) - call allocate_if_necessary(self%sdbc , mold=inference_engine%biases_) + call allocate_if_necessary(self%dcdb , mold=neural_network%biases_) ! Gradient of cost function with respect with biases + call allocate_if_necessary(self%vdb , mold=neural_network%biases_) + call allocate_if_necessary(self%sdb , mold=neural_network%biases_) + call allocate_if_necessary(self%vdbc , mold=neural_network%biases_) + call allocate_if_necessary(self%sdbc , mold=neural_network%biases_) ! TODO: #if ! (F2023_LOCALITY || F2018_LOCALITY) ! then don't allocate a, z, and delta - call allocate_if_necessary(self%z , mold=inference_engine%biases_) - call allocate_if_necessary(self%delta, mold=inference_engine%biases_) + call allocate_if_necessary(self%z , mold=neural_network%biases_) + call allocate_if_necessary(self%delta, mold=neural_network%biases_) - associate(output_layer => ubound(inference_engine%nodes_,1)) - allocate(self%a(maxval(inference_engine%nodes_), input_layer:output_layer)) ! Activations + associate(output_layer => ubound(neural_network%nodes_,1)) + allocate(self%a(maxval(neural_network%nodes_), input_layer:output_layer)) ! Activations end associate contains diff --git a/src/inference_engine_m.f90 b/src/fiats_m.f90 similarity index 88% rename from src/inference_engine_m.f90 rename to src/fiats_m.f90 index 45a803ab7..4d09e40c7 100644 --- a/src/inference_engine_m.f90 +++ b/src/fiats_m.f90 @@ -1,15 +1,15 @@ ! Copyright (c), The Regents of the University of California ! Terms of use are as specified in LICENSE.txt -module inference_engine_m +module fiats_m !! Specify the user-facing modules, derived types, and type parameters use double_precision_file_m, only : double_precision_file_t use double_precision_string_m, only : double_precision_string_t use hyperparameters_m, only : hyperparameters_t use input_output_pair_m, only : input_output_pair_t, shuffle, write_to_stdout - use inference_engine_m_, only : inference_engine_t, unmapped_network_t use kind_parameters_m, only : default_real, double_precision use metadata_m, only : metadata_t use mini_batch_m, only : mini_batch_t + use neural_network_m, only : neural_network_t, unmapped_network_t use network_configuration_m, only : network_configuration_t use tensor_m, only : tensor_t use tensor_map_m, only : tensor_map_t @@ -17,4 +17,4 @@ module inference_engine_m use training_configuration_m, only : training_configuration_t use ubounds_m, only : ubounds_t implicit none -end module +end module fiats_m diff --git a/src/inference_engine/tmp b/src/inference_engine/tmp deleted file mode 100644 index e69de29bb..000000000 diff --git a/test/asymmetric_engine_test_m.F90 b/test/asymmetric_network_test_m.F90 similarity index 87% rename from test/asymmetric_engine_test_m.F90 rename to test/asymmetric_network_test_m.F90 index 522bff898..f25dbfdda 100644 --- a/test/asymmetric_engine_test_m.F90 +++ b/test/asymmetric_network_test_m.F90 @@ -1,7 +1,7 @@ ! Copyright (c), The Regents of the University of California ! Terms of use are as specified in LICENSE.txt -module asymmetric_engine_test_m - !! Define asymmetric tests and procedures required for reporting results +module asymmetric_network_test_m + !! Define tests in which the desired output depends asymmetrically on the inputs ! External dependencies use assert_m, only : assert @@ -9,14 +9,14 @@ module asymmetric_engine_test_m test_t, test_result_t, vector_test_description_t, test_description_substring, string_t, vector_function_strategy_t ! Internal dependencies - use inference_engine_m, only : inference_engine_t, tensor_t + use fiats_m, only : neural_network_t, tensor_t implicit none private - public :: asymmetric_engine_test_t + public :: asymmetric_network_test_t - type, extends(test_t) :: asymmetric_engine_test_t + type, extends(test_t) :: asymmetric_network_test_t contains procedure, nopass :: subject procedure, nopass :: results @@ -31,7 +31,7 @@ module asymmetric_engine_test_m pure function subject() result(specimen) character(len=:), allocatable :: specimen - specimen = "An inference_engine_t object encoding an asymmetric XOR-AND-the-2nd-input network" + specimen = "An neural_network_t object encoding an asymmetric XOR-AND-the-2nd-input network" end function function results() result(test_results) @@ -60,8 +60,8 @@ function results() result(test_results) end associate end function - function xor_and_2nd_input_network() result(inference_engine) - type(inference_engine_t) inference_engine + function xor_and_2nd_input_network() result(neural_network) + type(neural_network_t) neural_network real, allocatable :: biases(:,:), weights(:,:,:) type(string_t), allocatable :: metadata(:) integer, parameter :: n(0:*) = [2,4,4,1] @@ -107,14 +107,14 @@ function xor_and_2nd_input_network() result(inference_engine) biases(1:n(l),l) = [-1] call assert(j == n(l), "l=3; j=1: j == n(l)") - inference_engine = inference_engine_t(metadata, weights, biases, nodes = n) + neural_network = neural_network_t(metadata, weights, biases, nodes = n) end function function xor_and_2nd_input_truth_table() result(test_passes) logical, allocatable :: test_passes(:) - type(inference_engine_t) asymmetric + type(neural_network_t) asymmetric asymmetric = xor_and_2nd_input_network() @@ -144,4 +144,4 @@ function xor_and_2nd_input_truth_table() result(test_passes) end function -end module asymmetric_engine_test_m +end module asymmetric_network_test_m diff --git a/test/hyperparameters_test_m.F90 b/test/hyperparameters_test_m.F90 index 2313c1c91..07d1140ab 100644 --- a/test/hyperparameters_test_m.F90 +++ b/test/hyperparameters_test_m.F90 @@ -4,7 +4,7 @@ module hyperparameters_test_m !! Test hyperparameters_t object I/O and construction ! External dependencies - use inference_engine_m, only : hyperparameters_t + use fiats_m, only : hyperparameters_t use julienne_m, only : test_t, test_result_t, test_description_t, test_description_substring, string_t #ifdef __GFORTRAN__ use julienne_m, only : test_function_i diff --git a/test/main.F90 b/test/main.F90 index a21444d24..40026b524 100644 --- a/test/main.F90 +++ b/test/main.F90 @@ -1,8 +1,8 @@ ! Copyright (c), The Regents of the University of California ! Terms of use are as specified in LICENSE.txt program main - use inference_engine_test_m, only : inference_engine_test_t - use asymmetric_engine_test_m, only : asymmetric_engine_test_t + use neural_network_test_m, only : neural_network_test_t + use asymmetric_network_test_m, only : asymmetric_network_test_t use trainable_network_test_m, only : trainable_network_test_t use metadata_test_m, only : metadata_test_t use hyperparameters_test_m, only : hyperparameters_test_t @@ -13,8 +13,8 @@ program main use julienne_m, only : command_line_t implicit none - type(inference_engine_test_t) inference_engine_test - type(asymmetric_engine_test_t) asymmetric_engine_test + type(neural_network_test_t) neural_network_test + type(asymmetric_network_test_t) asymmetric_network_test type(trainable_network_test_t) trainable_network_test type(hyperparameters_test_t) hyperparameters_test type(metadata_test_t) metadata_test @@ -47,8 +47,8 @@ program main call training_configuration_test%report(passes, tests) call tensor_map_test%report(passes, tests) call tensor_test%report(passes, tests) - call asymmetric_engine_test%report(passes, tests) - call inference_engine_test%report(passes, tests) + call asymmetric_network_test%report(passes, tests) + call neural_network_test%report(passes, tests) call trainable_network_test%report(passes, tests) call cpu_time(t_finish) diff --git a/test/metadata_test_m.F90 b/test/metadata_test_m.F90 index b87b499b3..5c3448a70 100644 --- a/test/metadata_test_m.F90 +++ b/test/metadata_test_m.F90 @@ -4,7 +4,7 @@ module metadata_test_m !! Test metadata_t object I/O and construction ! External dependencies - use inference_engine_m, only : metadata_t + use fiats_m, only : metadata_t use julienne_m, only : test_t, test_result_t, test_description_t, test_description_substring, string_t #ifdef __GFORTRAN__ use julienne_m, only : test_function_i diff --git a/test/network_configuration_test_m.F90 b/test/network_configuration_test_m.F90 index 97e4e8a64..5ba09fc97 100644 --- a/test/network_configuration_test_m.F90 +++ b/test/network_configuration_test_m.F90 @@ -4,7 +4,7 @@ module network_configuration_test_m !! Test network_configuration_t object I/O and construction ! External dependencies - use inference_engine_m, only : network_configuration_t + use fiats_m, only : network_configuration_t use julienne_m, only : test_t, test_result_t, test_description_t, test_description_substring, string_t, file_t #ifdef __GFORTRAN__ use julienne_m, only : test_function_i diff --git a/test/inference_engine_test_m.F90 b/test/neural_network_test_m.F90 similarity index 82% rename from test/inference_engine_test_m.F90 rename to test/neural_network_test_m.F90 index 2d7d3071c..0207ca7d1 100644 --- a/test/inference_engine_test_m.F90 +++ b/test/neural_network_test_m.F90 @@ -1,6 +1,6 @@ ! Copyright (c), The Regents of the University of California ! Terms of use are as specified in LICENSE.txt -module inference_engine_test_m +module neural_network_test_m !! Define inference tests and procedures required for reporting results ! External dependencies @@ -12,14 +12,14 @@ module inference_engine_test_m #endif ! Internal dependencies - use inference_engine_m, only : inference_engine_t, tensor_t, metadata_t + use fiats_m, only : neural_network_t, tensor_t, metadata_t implicit none private - public :: inference_engine_test_t + public :: neural_network_test_t - type, extends(test_t) :: inference_engine_test_t + type, extends(test_t) :: neural_network_test_t contains procedure, nopass :: subject procedure, nopass :: results @@ -29,7 +29,7 @@ module inference_engine_test_m pure function subject() result(specimen) character(len=:), allocatable :: specimen - specimen = "An inference_engine_t that encodes an XOR gate" + specimen = "An neural_network_t that encodes an XOR gate" end function function results() result(test_results) @@ -74,12 +74,12 @@ function results() result(test_results) test_results = test_descriptions%run() end function - function single_hidden_layer_xor_network() result(inference_engine) - type(inference_engine_t) inference_engine + function single_hidden_layer_xor_network() result(neural_network) + type(neural_network_t) neural_network integer, parameter :: nodes_per_layer(*) = [2, 3, 1] integer, parameter :: max_n = maxval(nodes_per_layer), layers = size(nodes_per_layer) - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = [string_t("XOR"), string_t("Damian Rouson"), string_t("2023-07-02"), string_t("step"), string_t("false")], & weights = reshape([real:: [1,1,0, 0,1,1, 0,0,0], [1,0,0, -2,0,0, 1,0,0]], [max_n, max_n, layers-1]), & biases = reshape([[0.,-1.99,0.], [0., 0., 0.]], [max_n, layers-1]), & @@ -87,12 +87,12 @@ function single_hidden_layer_xor_network() result(inference_engine) ) end function - function multi_layer_xor_network() result(inference_engine) - type(inference_engine_t) inference_engine + function multi_layer_xor_network() result(neural_network) + type(neural_network_t) neural_network integer, parameter :: nodes_per_layer(*) = [2, 3, 3, 1] integer, parameter :: max_n = maxval(nodes_per_layer), layers = size(nodes_per_layer) - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = [string_t("XOR"), string_t("Damian Rouson"), string_t("2023-07-02"), string_t("step"), string_t("false")], & weights = reshape([real:: [1,1,0, 0,1,1, 1,0,0, 1,0,0, 0,1,0, 0,0,1], [1,0,0, -2,0,0, 1,0,0]], & [max_n, max_n, layers-1]), & @@ -101,13 +101,13 @@ function multi_layer_xor_network() result(inference_engine) ) end function - function decrement_split_combine_increment() result(inference_engine) + function decrement_split_combine_increment() result(neural_network) !! Define a network that produces outputs identical to the 2 inputs for any input greater than or equal to 1 !! based on the following algorithm: !! 1. A 1st hidden layer that forwards input 1 unmolested and decrements input 2 by 1, !! 2. A 2nd hidden layer that forwards input 1 unmolested and splits input 2 into two halves, !! 3. An output layer that recombines those two halves and increments the result by 1. - type(inference_engine_t) inference_engine + type(neural_network_t) neural_network integer, parameter :: inputs = 2, hidden(*) = [2,3], outputs = 2 ! number of neurons in input, output, and hidden layers integer, parameter :: n(*) = [inputs, hidden(1), hidden(2), outputs] ! nodes per layer integer, parameter :: n_max = maxval(n), layers=size(n) ! max layer width, number of layers @@ -116,7 +116,7 @@ function decrement_split_combine_increment() result(inference_engine) w(*,*,*) = reshape( [1.,0.,0., 0.,1.,0., 0.,0.,0., 1.,0.,0., 0.,.5,.5, 0.,0.,0., 1.,0.,0., 0.,1.,0., 0.,1.,1.], w_shape), & b(*,*) = reshape( [0.,-1.,0., 0.,0.,0., 0.,1.,0.], b_shape) - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = [ & string_t("Decrement/Split/Combine/Increment => Identity") & ,string_t("Damian Rouson") & @@ -127,9 +127,9 @@ function decrement_split_combine_increment() result(inference_engine) ) end function - function double_precision_network() result(inference_engine) + function double_precision_network() result(neural_network) !! The result is a double-precision version of the this network defined in the decrement_split_combine_increment function - type(inference_engine_t(double_precision)) inference_engine + type(neural_network_t(double_precision)) neural_network integer, parameter :: inputs = 2, hidden(*) = [2,3], outputs = 2 ! number of neurons in input, output, and hidden layers integer, parameter :: n(*) = [inputs, hidden(1), hidden(2), outputs] ! nodes per layer integer, parameter :: n_max = maxval(n), layers=size(n) ! max layer width, number of layers @@ -139,15 +139,15 @@ function double_precision_network() result(inference_engine) [double precision :: 1.,0.,0., 0.,1.,0., 0.,0.,0., 1.,0.,0., 0.,.5,.5, 0.,0.,0., 1.,0.,0., 0.,1.,0., 0.,1.,1.], w_shape & ), b(*,*) = reshape( [double precision :: 0.,-1.,0., 0.,0.,0., 0.,1.,0.], b_shape) - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = metadata_t( & string_t("Double-Precision"), string_t("Damian Rouson"), string_t("2024-09-02"), string_t("relu"), string_t("false") & ), weights = w, biases = b, nodes = n & ) end function - function varying_width() result(inference_engine) - type(inference_engine_t) inference_engine + function varying_width() result(neural_network) + type(neural_network_t) neural_network integer, parameter :: inputs = 2, hidden(*) = [2,3], outputs = 2 ! number of neurons in input, output, and hidden layers integer, parameter :: n(*) = [inputs, hidden(1), hidden(2), outputs] ! nodes per layer integer, parameter :: n_max = maxval(n), layers=size(n) ! max layer width, number of layers @@ -158,14 +158,14 @@ function varying_width() result(inference_engine) w = reshape( [(i, i=1,product(w_shape))], w_shape) b = reshape( [(maxval(w) + i, i=1,product(b_shape))], b_shape) - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = [string_t("random"), string_t("Damian Rouson"), string_t("2024-07-03"), string_t("sigmoid"), string_t("false")], & weights = w, biases = b, nodes = n & ) end function - function distinct_parameters() result(inference_engine) - type(inference_engine_t) inference_engine + function distinct_parameters() result(neural_network) + type(neural_network_t) neural_network integer, parameter :: inputs = 2, hidden = 3, outputs = 1 ! number of neurons in input, output, and hidden layers integer, parameter :: n(*) = [inputs, hidden, hidden, outputs] ! nodes per layer integer, parameter :: n_max = maxval(n), layers=size(n) ! max layer width, number of layers @@ -176,7 +176,7 @@ function distinct_parameters() result(inference_engine) w = reshape( [(i, i=1,product(w_shape))], w_shape) b = reshape( [(maxval(w) + i, i=1,product(b_shape))], b_shape) - inference_engine = inference_engine_t( & + neural_network = neural_network_t( & metadata = [string_t("random"), string_t("Damian Rouson"), string_t("2023-07-15"), string_t("sigmoid"), string_t("false")], & weights = w, biases = b, nodes = n & ) @@ -184,54 +184,54 @@ function distinct_parameters() result(inference_engine) function multi_hidden_layer_net_to_from_json() result(test_passes) logical test_passes - type(inference_engine_t) inference_engine, from_json + type(neural_network_t) neural_network, from_json type(file_t) json_file - inference_engine = distinct_parameters() - json_file = inference_engine%to_json() - from_json = inference_engine_t(json_file) - test_passes = inference_engine == from_json + neural_network = distinct_parameters() + json_file = neural_network%to_json() + from_json = neural_network_t(json_file) + test_passes = neural_network == from_json end function function varying_width_net_to_from_json() result(test_passes) logical test_passes - associate(inference_engine => varying_width()) - associate(from_json => inference_engine_t( inference_engine%to_json() )) - test_passes = inference_engine == from_json + associate(neural_network => varying_width()) + associate(from_json => neural_network_t( neural_network%to_json() )) + test_passes = neural_network == from_json end associate end associate end function function infer_with_varying_width_net() result(test_passes) logical test_passes - type(inference_engine_t) inference_engine + type(neural_network_t) neural_network type(tensor_t) inputs, outputs real, parameter :: tolerance = 1.E-08 - inference_engine = decrement_split_combine_increment() + neural_network = decrement_split_combine_increment() inputs = tensor_t([1.1, 2.7]) - outputs = inference_engine%infer(inputs) + outputs = neural_network%infer(inputs) test_passes = all(abs(inputs%values() - outputs%values()) < tolerance) end function function double_precision_inference() result(test_passes) logical test_passes - type(inference_engine_t(double_precision)) inference_engine + type(neural_network_t(double_precision)) neural_network type(tensor_t(double_precision)) inputs, outputs real, parameter :: tolerance = 1.D-08 - inference_engine = double_precision_network() + neural_network = double_precision_network() inputs = tensor_t([1.1D0, 2.7D0]) - outputs = inference_engine%infer(inputs) + outputs = neural_network%infer(inputs) test_passes = all(abs(inputs%values() - outputs%values()) < tolerance) end function function elemental_infer_with_1_hidden_layer_xor_net() result(test_passes) logical test_passes - type(inference_engine_t) inference_engine + type(neural_network_t) neural_network - inference_engine = single_hidden_layer_xor_network() + neural_network = single_hidden_layer_xor_network() block type(tensor_t), allocatable :: truth_table(:) @@ -239,7 +239,7 @@ function elemental_infer_with_1_hidden_layer_xor_net() result(test_passes) integer i associate(array_of_inputs => [tensor_t([true,true]), tensor_t([true,false]), tensor_t([false,true]), tensor_t([false,false])]) - truth_table = inference_engine%infer(array_of_inputs) + truth_table = neural_network%infer(array_of_inputs) end associate test_passes = all( & abs(truth_table(1)%values() - false) < tolerance .and. abs(truth_table(2)%values() - true) < tolerance .and. & @@ -250,9 +250,9 @@ function elemental_infer_with_1_hidden_layer_xor_net() result(test_passes) function elemental_infer_with_2_hidden_layer_xor_net() result(test_passes) logical test_passes - type(inference_engine_t) inference_engine + type(neural_network_t) neural_network - inference_engine = multi_layer_xor_network() + neural_network = multi_layer_xor_network() block type(tensor_t), allocatable :: truth_table(:) @@ -260,7 +260,7 @@ function elemental_infer_with_2_hidden_layer_xor_net() result(test_passes) integer i associate(array_of_inputs => [tensor_t([true,true]), tensor_t([true,false]), tensor_t([false,true]), tensor_t([false,false])]) - truth_table = inference_engine%infer(array_of_inputs) + truth_table = neural_network%infer(array_of_inputs) end associate test_passes = all( & abs(truth_table(1)%values() - false) < tolerance .and. abs(truth_table(2)%values() - true) < tolerance .and. & @@ -269,4 +269,4 @@ function elemental_infer_with_2_hidden_layer_xor_net() result(test_passes) end block end function -end module inference_engine_test_m +end module neural_network_test_m diff --git a/test/tensor_map_test_m.F90 b/test/tensor_map_test_m.F90 index b99c31d3b..cdc961411 100644 --- a/test/tensor_map_test_m.F90 +++ b/test/tensor_map_test_m.F90 @@ -6,7 +6,7 @@ module tensor_map_test_m ! External dependencies use assert_m, only : assert use julienne_m, only : string_t, test_t, test_result_t, test_description_t, test_description_substring, file_t - use inference_engine_m, only : tensor_map_t, tensor_t + use fiats_m, only : tensor_map_t, tensor_t #ifdef __GFORTRAN__ use julienne_m, only : test_function_i #endif diff --git a/test/tensor_test_m.f90 b/test/tensor_test_m.f90 index f7e321504..7d4ac7c21 100644 --- a/test/tensor_test_m.f90 +++ b/test/tensor_test_m.f90 @@ -11,7 +11,7 @@ module tensor_test_m #endif ! Internal dependencies - use inference_engine_m, only : tensor_t + use fiats_m, only : tensor_t implicit none diff --git a/test/trainable_network_test_m.F90 b/test/trainable_network_test_m.F90 index 92518d74d..f6c88d910 100644 --- a/test/trainable_network_test_m.F90 +++ b/test/trainable_network_test_m.F90 @@ -12,7 +12,7 @@ module trainable_network_test_m #endif ! Internal dependencies - use inference_engine_m, only : trainable_network_t, inference_engine_t, tensor_t, input_output_pair_t, mini_batch_t, shuffle + use fiats_m, only : trainable_network_t, neural_network_t, tensor_t, input_output_pair_t, mini_batch_t, shuffle implicit none private @@ -183,7 +183,7 @@ function two_zeroed_hidden_layers() result(trainable_network) w = 0. b = 0. - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = neurons, weights = w, biases = b & ,metadata = [string_t("2-hide|3-wide"), string_t("Rouson"), string_t("2023-06-30"), string_t("sigmoid"), string_t("false")] & )) @@ -201,7 +201,7 @@ function two_random_hidden_layers() result(trainable_network) b = 2*b w = 2*w - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = neurons, weights = w, biases = b & ,metadata = [string_t("2-hide|3-wide"), string_t("Rouson"), string_t("2023-06-30"), string_t("sigmoid"), string_t("false")] & )) @@ -420,7 +420,7 @@ function perturbed_identity_network(perturbation_magnitude) result(trainable_net call random_number(harvest) harvest = perturbation_magnitude*harvest - trainable_network = trainable_network_t( inference_engine_t( & + trainable_network = trainable_network_t( neural_network_t( & nodes = nodes_per_layer, & weights = identity + harvest , & biases = reshape([real:: [0,0], [0,0], [0,0]], [max_n, layers-1]), & diff --git a/test/training_configuration_test_m.F90 b/test/training_configuration_test_m.F90 index 13ed5ba2d..a09dc0c09 100644 --- a/test/training_configuration_test_m.F90 +++ b/test/training_configuration_test_m.F90 @@ -4,7 +4,7 @@ module training_configuration_test_m !! Test training_configuration_t object I/O and construction ! External dependencies - use inference_engine_m, only : training_configuration_t, hyperparameters_t, network_configuration_t + use fiats_m, only : training_configuration_t, hyperparameters_t, network_configuration_t use julienne_m, only : test_t, test_result_t, test_description_t, test_description_substring, string_t, file_t #ifdef __GFORTRAN__ use julienne_m, only : test_function_i