First pass at introduction. Added latexmk files.

.gitignore

@@ -0,0 +1 @@
+output/

chapters/episodic_ast.tex

@@ -282,12 +282,12 @@ \subsection{Interface}\label{sec:implementation_interface}
     \toprule
     \textbf{Function} & \textbf{$\bm{\text{Input}\mapsto\text{Output}}$} \\
     \midrule
-    \textsc{Initialize} & $\bar{S} \mapsto \emptyset$ \\
-    \textsc{Evaluate} & $\bar{S} \mapsto \langle p, e, d \rangle$ \\
-    $\quad$\textsc{Transition} & $\bar{S} \mapsto p \in \mathbb{R}$ \\
-    $\quad$\textsc{MissDistance} & $\bar{S} \mapsto d \in \mathbb{R}$ \\
-    $\quad$\textsc{IsEvent} & $\bar{S} \mapsto e \in \mathbb{B}$ \\
-    \textsc{IsTerminal} & $\bar{S} \mapsto \tau \in \mathbb{B}$ \\
+    \textsc{Initialize} & $\bar{\mathcal{S}} \mapsto \emptyset$ \\
+    \textsc{Evaluate} & $\bar{\mathcal{S}} \mapsto \langle p, e, d \rangle$ \\
+    $\quad$\textsc{Transition} & $\bar{\mathcal{S}} \mapsto p \in \mathbb{R}$ \\
+    $\quad$\textsc{MissDistance} & $\bar{\mathcal{S}} \mapsto d \in \mathbb{R}$ \\
+    $\quad$\textsc{IsEvent} & $\bar{\mathcal{S}} \mapsto e \in \mathbb{B}$ \\
+    \textsc{IsTerminal} & $\bar{\mathcal{S}} \mapsto \tau \in \mathbb{B}$ \\
     \bottomrule
   \end{tabular}
   \end{threeparttable}

chapters/introduction.tex

@@ -1,6 +1,64 @@
+% - shortest intro chapter possible (concise)
+% - "freshman undergrad could understand contributions"
 
-\subsection{Motivation}
+% \section{Motivation}
+With the expanding use of artificial intelligent algorithms to solve complex problems, there is an increasing need to ensure the validity of such systems. % safety?
+It may be challenging to exhaustively validate a complex system due to the continuous nature of systems deployed in the real-world.
+Therefore, to validate such systems, we need to rely on simulation.
+As with all simulations, we model the real-world through a series of approximations.
+Despite these approximations, it may still be computationally difficult to find system failures efficiently.
+The validation problem becomes even more challenging when failures are extremely rare and where a simple random search may severly underestimate the probability of failure (or even estimate it as zero).
+This work purposes several methods to try and efficiently search for likely failures in safety-critical systems, modeling the system as a black box.
 
-\subsection{Black-Box Systems Definition}
 
-\subsection{Safety-Critical Applications}
+The term \textit{black box} refers to a software system for which we pass inputs and only have access to the provided outputs (i.e., the internals of the black-box system are unknown to us).
+Framing the problem around black-box systems allows these techiniques to be broadly applied to existing systems without the need to gain access to the internal code itself. 
+Techniques that require knowledge of the system internals are termed \textit{white box}, which are not the focus of this work. 
+% - Gray-box
+% - Safety-critical systems?
+% - Cyber-physical systems?
+
+The validation problem can be split into \textit{falsification} (i.e., finding failures) and \textit{most-likely failure analysis} (i.e., finding likely failures). 
+% We propose a falsification approach to 
+% - CE-surrogate: what about it is used for rare-failure finding?
+% - How we use surrogate models to offload the computation to a learned function approximator.
+% - Sierra represents the failures of a system. We want an algorithm to find the most-likely failure given that the failures are rare. CEM is a method to do this, yet requires many calls to the underlying system to evaluate the goodness of the search. We propose a surrogate modeling-based CEM algorithm that outperforms CEM using the same number of system evaluations.
+We consider two types of systems we want to validate: sequential decision making systems and non-sequential classification systems (e.g., a hand-written digit classifier).
+Each system type has their own respective challenges to consider.
+We are also motivated by complex systems that are computationally expensive to evaluate, thus introducing techniques to intelligently select when we execute the system to reduce unnecessary evaluations. 
+
+% The core of this work is built on the \textit{adaptive stress testing} problem formulation for finding likely failures in black-box systems \cite{lee2020adaptive}.
+
+
+\section{Contributions} % (fold)
+\label{sec:contributions}
+
+The technical contributions of this thesis consist of the following:
+
+\begin{itemize}
+    \item Cross-entropy surrogate method; algorithm for surrogate-based black-box stochastic optimization. Including a novel optimization test function (sierra).
+    \item Episodic adaptive stress testing formulation.
+    \item Framework for validation of large, static datasets using adversarial weakness recognition.
+    \item Open source tooling to apply general black-box adaptive stress testing (POMDPStressTesting.jl), the cross-entropy method algorithm variants (CrossEntropyVariants.jl), and the adversarial weakness recognition (FailureRepresentation.jl).
+\end{itemize}
+
+% section contributions (end)
+
+% \section{Black-Box Systems Definition}
+
+% \section{Safety-Critical Applications}
+
+
+% 1) Introduced varients of the cross-entropy method algorithm using surrogate models, mixture models, and sub-routine calls to the original CE-method (used to find rare events over computationally expensive systems)
+%     - Open sourced tooling: CrossEntropyVariants.jl
+
+% 2) Reformulation of adaptive stress testing (AST) and MCTS to be applied to computationally expensive episodic problems
+%     - With an application to aircraft trajectory predictions
+%     - Important: will cover tooling in a moment. Open sourced tooling, general: POMDPStressTesting.jl
+
+% 3) For non-sequential systems (think MNIST classifier), I developed a framework for selectively validating large datasets (where exhaustive search would be too computationally expensive given minor changes to the system)
+%     - This work is currently in review for an NSF grant (joint with Berkeley) and a SystemX proposal (under the Autonomous Systems category)
+%     - Open sourced tooling: FailureRepresentation.jl
+
+% 4) Tooling: POMDPStressTesting.jl package
+%     - Open source tooling to apply AST work to general black-box systems

diagrams/ast-formulation.tex

@@ -33,7 +33,7 @@
 
     \begin{pgfonlayer}{simlayer}
         \path let \p1=(wptdist.north west), \p2=(sut.south east) in node (sim) [fit={($(\x1,\y1)+(-4cm,1.25cm)$) ($(\x2,\y2)+(1.3cm,-0.5cm)$)},simstyle,line width=0.08cm]{};
-        \node at ($(sim.north west)+(2cm,-7mm)$) [font={\LARGE}] {Simulator $\mathcal{\bar{S}}$};
+        \node at ($(sim.north west)+(2cm,-7mm)$) [font={\Large}] {Simulator $\bar{\mathcal{S}}$};
     \end{pgfonlayer}
 
     \draw[->,dashed,line width=0.5pt] (wptdist.south) |- ($(sim.west)+(0mm,-1cm)$) node[pos=0.5,xshift=-3cm,above]{transition probability $p$};

diagrams/standard-ast-formulation.tex

@@ -31,7 +31,7 @@
 
     \begin{pgfonlayer}{simlayer}
         \path let \p1=(wptdist.north west), \p2=(sut.south east) in node (sim) [fit={($(\x1,\y1)+(-0.5cm,1.25cm)$) ($(\x2,\y2)+(0.5cm,-0.5cm)$)},simstyle,line width=0.08cm]{};
-        \node at ($(sim.north west)+(2cm,-7mm)$) [font={\LARGE}] {Simulator $\mathcal{\bar{S}}$};
+        \node at ($(sim.north west)+(2cm,-7mm)$) [font={\Large}] {Simulator $\bar{\mathcal{S}}$};
     \end{pgfonlayer}
 
     \node (learner) [learnerstyle, above left=0.25cm and 7.25cm of sim.west,xshift=0cm] {Reinforcement\\Learner};

latexmkrc

@@ -0,0 +1,9 @@
+ensure_path( 'BIBINPUTS', './references//' );
+
+$pdf_mode = 4;
+$out_dir = 'output';
+$aux_dir = 'output';
+
+$lualatex = "lualatex %O -shell-escape %S";
+
+@default_files = ('main');

main.tex

@@ -21,6 +21,9 @@
 \addbibresource{references/weakness_rec.bib}
 \addbibresource{references/pomdpstresstesting.bib}
 
+% DRAFTING: speed up compilation.
+% \def\includebody{} % sets to true
+\let\includebody\undefined % sets to false
 
 \begin{document}
 \title{Algorithms for efficient validation of black-box systems}
@@ -38,25 +41,29 @@
 \chapter{Introduction}
 \input{chapters/introduction}
 
-% Cross-Entropy Method Variants for Optimization
-\chapter{Cross-Entropy Surrogate Method} % TODO: change
-\input{chapters/cem_variants}
+\ifdefined\includebody
+    % Cross-Entropy Method Variants for Optimization
+    \chapter{Cross-Entropy Surrogate Method} % TODO: change
+    \input{chapters/cem_variants}
 
-\chapter{Episodic Adaptive Stress Testing}
-\input{chapters/episodic_ast}
+    \chapter{Episodic Adaptive Stress Testing}
+    \input{chapters/episodic_ast}
 
-\chapter{Adversarial Weakness Recognition}
-\input{chapters/weakness_rec}
+    \chapter{Adversarial Weakness Recognition}
+    \input{chapters/weakness_rec}
 
-% POMDPStressTesting.jl: Adaptive Stress Testing for Black-Box Systems
-\chapter{POMDPStressTesting.jl}
-\input{chapters/pomdpstresstesting}
+    % POMDPStressTesting.jl: Adaptive Stress Testing for Black-Box Systems
+    \chapter{POMDPStressTesting.jl}
+    \input{chapters/pomdpstresstesting}
+\fi
 
 \chapter{Conclusions}
 ...
 
 \appendix
-\input{appendices/episodic_ast_appendix}
+\ifdefined\includebody
+    \input{appendices/episodic_ast_appendix}
+\fi
 
 % \bibliographystyle{plain}
 % \bibliography{cem_variants}

preamble/preamble.tex

@@ -144,7 +144,7 @@
 % \renewcommand{\bibfont}{\footnotesize} % for IEEE bibfont size TODO.
 \usepackage{multirow}
 \usepackage{bbm}
-\usepackage{hyperref}
+\usepackage[hidelinks]{hyperref}
 \usepackage{cleveref}
 % \usepackage[keeplastbox]{flushend} % TODO.
 

references/pomdpstresstesting.bib

@@ -155,4 +155,13 @@ @article{gym
   author={Brockman, Greg and Cheung, Vicki and Pettersson, Ludwig and Schneider, Jonas and Schulman, John and Tang, Jie and Zaremba, Wojciech},
   journal={arXiv:1606.01540},
   year={2016}
-}
+}
+
+@article{lee2020adaptive,
+    title = {Adaptive Stress Testing: Finding Likely Failure Events with Reinforcement Learning},
+    author = {Lee, Ritchie and Mengshoel, Ole J. and Saksena, Anshu and Gardner, Ryan and Genin, Daniel and Silbermann, Joshua and Owen, Michael and Kochenderfer, Mykel J.},
+    journal = jair,
+    year = {2020},
+    volume = {69},
+    pages = {1165--1201}
+}

references/weakness_rec.bib

@@ -1646,4 +1646,4 @@ @article{kramer1991nonlinear
   pages={233--243},
   year={1991},
   publisher={Wiley Online Library}
-}
+}