RESEARCH_NOTES.txt

https://gallery.rcpp.org/articles/first-steps-with-C++11/
// Enable C++11 via this plugin (Rcpp 0.10.3 or later)
// [[Rcpp::plugins(cpp11)]]



Rcpp quick ref
http://dirk.eddelbuettel.com/code/rcpp/Rcpp-quickref.pdf

Rcpp books
!!! Rcpp for everyone (2019)
    https://teuder.github.io/rcpp4everyone_en/
    https://github.com/teuder/rcpp4everyone_en
! R packages by Hadley Wickham > Compiled code
  http://r-pkgs.had.co.nz/src.html
! Advance R by Hadley >
  http://adv-r.had.co.nz/Rcpp.html


Rcpp tutorials
! Expose a CPP class to R via Rcpp
  https://www.gormanalysis.com/blog/exposing-a-cpp-student-class-with-rcpp/

Rcpp source code:
https://github.com/RcppCore/Rcpp



Debugging Rcpp with gdb on Windows 8 !!!
http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2014-September/008036.html
C:\Rtools\gcc-4.6.3\bin64\gdb.exe rgui
https://stackoverflow.com/questions/53622354/how-to-debug-line-by-line-rcpp-generated-code-in-windows
https://stackoverflow.com/questions/11345537/debugging-line-by-line-of-rcpp-generated-dll-under-windows


Videos:

Using gdb to debug R packages with native code
https://vimeo.com/11937905


gdb tutorials and cheat sheets:

http://www.yolinux.com/TUTORIALS/GDB-Commands.html
https://lldb.llvm.org/use/map.html#examining-variables


Debug output of Rcpp types (always working despite template and inline behaviour of C++ compilers)
- but may crash due to minimal precond checks -
call Rf_PrintValue(x->data)



R source code (mirrored):
https://github.com/wch/r-source/tree/trunk/src/include


R Docs:
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Handling-R-objects-in-C
https://cran.r-project.org/doc/manuals/r-release/R-ints.html#R-Internal-Structures



# Compile flags:
usethis::edit_r_makevars()
# Or (one off) Go to the directory ~/.R (a hidden directory with the .).
# Create a new file called "Makevars" and in it add the line CXXFLAGS=-g -O0 -Wall.
#
# Folder:
# HOME/.R/Makevars-R_PLATFORM (but HOME/.R/Makevars.win or HOME/.R/Makevars.win64 on Windows),
# or if that does not exist, HOME/.R/Makevars, where ‘R_PLATFORM’ is the platform for which R was built



$ R CMD config CXXFLAGS
-g -O2 -Wall -pedantic -fstack-protector-strong -D_FORTIFY_SOURCE=2
CXXFLAGS -g -O0 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2

# debugging optimized code:
-ggdb3 -gdwarf-4 -fvar-tracking-assignments -ginline-points -gstatement-frontiers
Or just
-g3

# https://stackoverflow.com/questions/30946735/gdb-not-loading-symbols-when-running-standalone-shared-library
# to export all symbols when compiling use:
gcc -ggdb3 -O0 -pie
you also need -rdynamic flag to force exporting all symbols from libdtest1.so
-shared overrides -pie. Remove -shared
readelf -d libdtest1.so | grep DEBUG
the output for a library with debugging symbols contains "(debug) 0x0"

List exported symbols of an ELF shared library:
readelf -Ws /path/to/libfoo.so
or (if you have GNU nm)
nm -D /path/to/libfoo.so
# readelf -Ws Rcpp.so | grep wrap
#   3228: 00000000000921ce    34 FUNC    WEAK   DEFAULT   12 _ZN4Rcpp8internal21wrap_dispatch_unknownINS_12RObject_ImplINS_15PreserveStorageEEEEEP7SEXPRECRKT_NS_6traits17integral_constantIbLb1EEE

provides macro information if you are using the DWARF debugging format, and specify the option -g3
DWARF is currently the most expressive and best supported debugging format in GDB.
# -gdwarf-3
# -gdwarf-4

# To avoid inlined functions use:
-fno-inline

# CFLAGS += -fkeep-inline-functions
# gcc -fkeep-inline-functions


# Sombody said position-independent code shall make templated code available in gdb... Really?:
-fPIC  # no, does not work (neither with -ggdb nor -gdwarf-4)



https://stackoverflow.com/questions/24957363/how-to-print-evaluate-c-template-functions-in-gdb
Without an explicit instance in the source code, the compiler will treat the template code
as it would "static inline" code and optimize it out if it is unused.

template<typename T>
T id(T x) {return x;}

template int id<int> (int x);

int main() {
  int i = 0;
  i = i + 1;
}

Within gdb, I place the C++ function (symbol) I want to call within single quotes:
(gdb) p 'id<int>(int)'(i)



# GNU GDB documentation:
https://www.gnu.org/software/gdb/documentation/

set print pretty on
set print demangle on
set print asm-demangle on
# add to ~/.gdbinit

# Use docker for valgrind debugging of memory issues
https://stackoverflow.com/questions/57360360/having-trouble-debugging-possible-rcpp-memory-leak
http://dirk.eddelbuettel.com/blog/2019/08/05/#023_rocker_debug_example


# !!! Debug Rcpp code in R on Windows (explains how to find and load a dynamically compiled CPP function!!!)
https://stackoverflow.com/questions/53622354/how-to-debug-line-by-line-rcpp-generated-code-in-windows
# Same for OS-X
https://stackoverflow.com/questions/21226337/what-are-productive-ways-to-debug-rcpp-compiled-code-loaded-in-r-on-os-x-maveri?rq=1
https://www.gormanalysis.com/blog/using-rcpp-in-xcode/



# R -d gdb --vanilla

# (gdb) run

# library(RcppTestPkg)

# Strg+C

# (gdb) break rcpp_hello_world.cpp:8  # :12

# (gdb) cont

# rcpp_hello_world()

# (gdb) p x
# $2 = {<Rcpp::PreserveStorage<Rcpp::Vector<16, Rcpp::PreserveStorage> >> = {
#   data = 0x5555562c4210}, <Rcpp::SlotProxyPolicy<Rcpp::Vector<16, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::AttributeProxyPolicy<Rcpp::Vector<16, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::NamesProxyPolicy<Rcpp::Vector<16, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::RObjectMethods<Rcpp::Vector<16, Rcpp::PreserveStorage> >> = {<No data fields>}, <Rcpp::VectorBase<16, true, Rcpp::Vector<16, Rcpp::PreserveStorage> >> = {<Rcpp::traits::expands_to_logical__impl<16>> = {<No data fields>}, <No data fields>}, cache = {
#     p = 0x7fffffffba10}}

# (gdb) p R_PV(x)
# 'R_PV' has unknown return type; cast the call to its declared return type
# (gdb) call R_PV(wrap(x))
# Cannot resolve function wrap to any overloaded instance
# call R_PV(Rcpp::wrap(&x))
# A syntax error in expression, near `(&x))'.
# (gdb) call R_PV('Rcpp::wrap'(&x))
# Cannot resolve function wrap to any overloaded instance
# (gdb) print x->data
# $2 = (SEXP) 0x555557332c10
# (gdb) call R_PV(x->data)
# (gdb) print R_PV(x->data)
# $3 = void
# !!!!!!!
# (gdb) call print(x->data)
# [1] "foo" "bar"
# (gdb) print print(x->data)
# [1] "foo" "bar"
# $5 = void
# (gdb) call Rf_PrintValue(x->data)
# [1] "foo" "bar"
# (gdb) call 'Rcpp::wrap<Rcpp::CharacterVector>'(x)
# No symbol "Rcpp::wrap<Rcpp::CharacterVector>" in current context.




# https://blogs.oracle.com/linux/8-gdb-tricks-you-should-know-v2
# (gdb) list rcpp_hello_world.cpp
# (gdb) directory path/to/source/code/files/


# Rcpp::wrap() and Rcpp::as():
# https://github.com/RcppCore/Rcpp/blob/f3c5a34e06e774532227470b01c63a8f08ce4313/inst/include/Rcpp/internal/wrap.h
# https://stackoverflow.com/questions/58652434/debugging-c-code-of-an-r-package-with-rcpp-using-gdb-cannot-print-variable-val?noredirect=1#comment103836387_58652434
# https://gallery.rcpp.org/articles/custom-as-and-wrap-example/
# use as<>() and especially wrap() to create yourself a SEXP
//conversion from C++ to R
template<typename T> SEXP wrap(const T& object) ;
//conversion from R to C++
template<typename T> T as( SEXP x) ;
#
# Transfer from R to C++, and from C++ to R
#
# R data types (SEXP) are matched to C++ objects in a class hierarchy. All R types are supported (vectors, functions, environment, etc ...) and each type is mapped to a dedicated class. For example, numeric vectors are represented as instances of the Rcpp::NumericVector class, environments are represented as instances of Rcpp::Environment, functions are represented as Rcpp::Function, etc ...
#
# The underlying C++ library also offers the Rcpp::wrap function which is a templated function that transforms an arbitrary object into a SEXP. This makes it straightforward to implement C++ logic in terms of standard C++ types such as STL containers and then wrap them when they need to be returned to R. Internally, wrap uses advanced template meta programming techniques and currently supports these data types: primitive types (bool, int, double,  size_t, Rbyte, Rcomplex, std::string), STL containers (e.g std::vector) where T is wrappable, STL maps (e.g std ::map) where T is wrappable, and arbitrary types that support implicit conversion to SEXP.
#
# The reverse conversion (from R into C++) is performed by the Rcpp::as function template offering a similar degree of flexibility.



# https://stackoverflow.com/questions/22029834/possible-to-call-inline-functions-in-gdb-and-or-emit-them-using-gcc
# For gdb to support inlined functions, the compiler must record information about inlining in the debug information:
# -gdwarf-2

# typedefs in Rcpp for xxx-Vectors in instantiation.h:
#     typedef Vector<REALSXP> NumericVector ;
#     typedef Vector<STRSXP> CharacterVector ;

# (gdb) call x.at(0)
# Cannot evaluate function -- may be inlined

#  provide the namespace that the class belongs to within a single quotes
# (gdb) p ('MyScope::MyClass'*) ptr;

# How to list class methods in gdb?
# ptype(x)

# Show function signature:
whatis Rcpp::wrap

# gdb attach fails with ptrace: Operation not permitted
# -> Ubuntu kernel hardening allows only debugging of child processes:
# https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#ptrace%20Protection
# To switch this off use:
# sudo su -
# echo 0 > /proc/sys/kernel/yama/ptrace_scope
# exit

# If you ask to print an object whose contents are unknown to GDB, e.g.,
# because its data type is not completely specified by the debug information,
# GDB will say ‘<incomplete type>’. See incomplete type, for more about this.

# https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Inspecting-R-objects
The key to inspecting R objects from compiled code is the function PrintValue(SEXP s)
which uses the normal R printing mechanisms to print the R object pointed to by s,
or the safer version R_PV(SEXP s) which will only print ‘objects’.
...
Another way is to call R_PV from the symbolic debugger. (PrintValue is hidden as Rf_PrintValue.)
See source code: https://github.com/wch/r-source/blob/9b732e3338ed0f618ba18d136a68ca68733de3ef/src/main/print.c
void R_PV(SEXP s)
{
    if(isObject(s)) PrintValueEnv(s, R_GlobalEnv);
}
=> R_PV ist nur sichtbar in gdb, wenn der Code die R-API benutzt:
   https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Handling-R-objects-in-C
   Rinternals.h contains public functions to handle R objects from within C code.
   Headers are in the directory R_INCLUDE_DIR (default R_HOME/include)
#include <R.h>
#include <Rinternals.h>

Oder: Über Rcpp einbinden (RcppCommon.h):

# Rf_PrintValue
# is declared in the header Rinternals.h that is pulled in by RcppCommon.h that is pulled in by Rcpp.h
# https://stackoverflow.com/questions/28152406/print-integer-vector-from-rcpp-function


(gdb) p R_PV('Rcpp::wrap'(y))
Cannot resolve function wrap to any overloaded instance
(gdb) p R_PV(Rcpp::wrap(y))
A syntax error in expression, near `(y))'.
(gdb) p 'Rcpp::wrap'(y)
Cannot resolve function wrap to any overloaded instance
(gdb) p Rcpp::wrap(y)
Cannot resolve function wrap to any overloaded instance
(gdb)



Access the global environment via Rcpp (code snippet):
https://gallery.rcpp.org/articles/accessing-environments/
https://stackoverflow.com/questions/3671862/read-variables-from-global-environment-with-inline-rcpp
https://teuder.github.io/rcpp4everyone_en/290_environment.html



# GDB pretty-printers in Python
https://sourceware.org/gdb/wiki/STLSupport
https://sourceware.org/gdb/onlinedocs/gdb/Writing-a-Pretty_002dPrinter.html#Writing-a-Pretty_002dPrinter
    Environment env = Environment::global_env();
    CharacterVector v = env["stooges"];
    Rcout << "Stooge Nb 2 is: " << v[1] << std::endl
    // STL containers should also work:
    // std::vector<double> vx = env["x"];
    // return wrap(vx);


Print R objects to stderr:
https://stackoverflow.com/questions/35982284/how-to-print-an-r-object-to-stderr-in-rcpp
https://stackoverflow.com/questions/51169994/how-to-print-raw-values-in-rcpp
-> REprintf in R
# src/include/R_ext/Print.h
void Rprintf(const char *, ...);
void REprintf(const char *, ...);
# r-source/src/main/printutils.c
-> All printing in R is done via the functions Rprintf and REprintf
   Rprintf writes to standard output
   REprintf writes to standard error
# See also: https://github.com/wch/r-source/blob/trunk/src/main/print.c
  #include <Rcpp.h>
  using namespace Rcpp;
  // [[Rcpp::export]]
  void test(SEXP key) {
    CharacterVector key1 = as<CharacterVector>(key);
    Rcpp::Rcerr << "This does not work: " << key1 << std::endl;
  }
  /*** R
  test(c("x", "y"))
  test(1:3)
    */


# Print helper function to print raw values
  void printVector(IntegerVector v) { print(v); }
  void printVector2(IntegerVector v) { Rcpp::Rcout << v << std::endl; }
https://stackoverflow.com/questions/51169994/how-to-print-raw-values-in-rcpp
  // [[Rcpp::export]]
  void print_raw(RawVector x) {
    for (int v : x) {
      Rcout << std::hex << std::setw(2) << std::setfill('0') << v << ' ';
    }
    Rcout << '\n';
  }
# or
  #include <Rcpp.h>
  using namespace Rcpp;

  // [[Rcpp::export]]
  void print_raw(RawVector x) {
    print(x);
  }

  /*** R
  x <- as.raw(0:10)
  print(x)
  print_raw(x)
  */


# https://stackoverflow.com/questions/7429462/creating-c-string-in-gdb
(gdb) call malloc(sizeof(std::string))
$1 = (void *) 0x91a6a0
(gdb) call ((std::string*)0x91a6a0)->basic_string()
(gdb) call ((std::string*)0x91a6a0)->assign("Hello, World")
$2 = (std::basic_string<char, std::char_traits<char>, std::allocator<char> > &) @0x91a6a0: {static npos = <optimized out>, _M_dataplus = {<std::allocator<char>> = {<__gnu_cxx::new_allocator<char>> = {<No data fields>}, <No data fields>}, _M_p = 0x91a6f8 "Hello, World"}}
(gdb) call SomeFunctionThatTakesAConstStringRef(*(const std::string*)0x91a6a0)



http://r-pkgs.had.co.nz/src.html

The default make file generated by R is located at file.path(R.home("etc"), "Makeconf").
# On Ubuntu:
# /usr/lib/R/etc/Makeconf

PKG_LIBS: Linker flags. A common use is PKG_LIBS = $(BLAS_LIBS). This allows you to use the same BLAS library as R.
PKG_CFLAGS & PKG_CXXFLAGS: C and C++ flags. Most commonly used to set define directives with -D.
PKG_CPPFLAGS: Pre-processor flags (not C++ flags!). Most commonly used to set include directories with -I

By default, R will use the system make, which is not always GNU compatible (i.e. on Solaris). If you want to use GNU extensions (which are extremely common), add SystemRequirements: GNU make to DESCRIPTION. If you’re not sure if you’re using GNU extensions, play it safe and add it to the system requirement.


If a package does have a src/Makevars[.win] file then setting the make variable ‘CXX_STD’ is preferred, as it allows R CMD SHLIB to work correctly in the package’s src directory.
CXX_STD = CXX11

Support for a C++14 compiler (where available) was been added to R from version 3.4.0.

Hence to use C++14 code in a package, the package’s Makevars file (or Makevars.win on Windows) should include the line

CXX_STD = CXX14

Note that g++ 4.9.x (as used for R on Windows at least up to 3.6.x) has only partial C++14 support, and the flag to obtain that support is not included in the default Windows build of R — one could try something like

CXX14="$(BINPREF)g++ $(M_ARCH)"
CXX14FLAGS="-O2 -Wall"
CXX14STD=-std=gnu1y

in HOME/.R/Makevars.win.

No C++17 support is enabled in the current default build of R on Windows.



You will also need to add // [[Rcpp::plugins(cpp11)]] at the top of your cpp file for std::unordered_map and auto.



Generic functions for R data types via run-time-dispatching via the RCPP_RETURN macros

-> RCPP_RETURN_VECTOR and RCPP_RETURN_MATRIX
   include/Rcpp/macros/dispatch.h

https://stackoverflow.com/questions/58965423/return-subset-of-a-given-sexp-without-knowing-the-actual-internal-data-type
https://stackoverflow.com/questions/19823915/how-can-i-handle-vectors-without-knowing-the-type-in-rcpp
https://stackoverflow.com/questions/55212746/rcpp-fast-statistical-mode-function-with-vector-input-of-any-type
https://stackoverflow.com/questions/16740338/what-is-the-r-to-c-syntax-for-vectors
https://stackoverflow.com/questions/46515052/extending-rcpp-function-to-input-vector-of-any-type

https://gallery.rcpp.org/articles/rcpp-return-macros/

https://gallery.rcpp.org/articles/rcpp-wrap-and-recurse/

https://stackoverflow.com/questions/55212746/rcpp-fast-statistical-mode-function-with-vector-input-of-any-type/55223792#55223792
-> Disable a non-supported RTYPE from the RCPP_RETURN macros:
     template <int RTYPE>
     Vector<RTYPE> fastModeImpl(Vector<RTYPE> x, bool narm) { ... }

     template <>
     Vector<CPLXSXP> fastModeImpl(Vector<CPLXSXP> x, bool narm) {
       stop("Not supported SEXP type!"); }

     // [[Rcpp::export]]
     SEXP fastMode( SEXP x, bool narm = false ) {
       RCPP_RETURN_VECTOR(fastModeImpl, x, narm); }




CPP non-type template parameters (Rcpp uses on-type parameter for the SEXP type, but also type parameters):

template<typename T, unsigned int size>
Array {
  // Implementation
};

Why would we use a non-type template parameter instead of a regular parameter?

The main difference is when the parameter is known to the program. Like all the template parameters and unlike the non-template parameters, the value must be known at compile time.

This is useful when we want to use the parameters in expressions that need to be evaluated at compile time.



rdoxygen: Create dosygen documentation for C++ code in R packages
-> can create an R vignette from the HTML documentation
https://github.com/mlysy/rdoxygen




Use gdb to examine a core file generated by a segfault:
https://bogotobogo.com/cplusplus/CppCrashDebuggingMemoryLeak.php


# Load a dynlib in lldb to inject debugging functions:
# target modules add loads the binary (and its dSYM, if there is one) into lldb's pool of binaries it knows about.
# But the dylib has not been loaded into your inferior process -- you will not be able to run the functions in there.
# (lldb) target modules add debugHelper.dylib  # does not work
# Instead use dlopen():
(lldb) p (int)dlopen("debugHelper.dylib", 10)
# (that 10 is RTLD_GLOBAL|RTLD_NOW, v. dlfcn.h).
# There's also an SB API method to do this, SBProcess::LoadImage
# which doesn't require you to look up the details of dlopen. e.g.
(lldb) script lldb.process.LoadImage(lldb.SBFileSpec("debugHelper.dylib", False), lldb.SBError())
# You can also use process load at the lldb command line.



Structuring a package embedding a C++ library containing cpp and hpp files
https://community.rstudio.com/t/structuring-a-package-embedding-a-c-library-containing-cpp-and-hpp-files/11696
I'm working on exposing a C++ library that contains cpp and corresponding hpp files (essentially 1 hpp per cpp file + meta hpp files that include other hpp files).
I'm wondering what the best / "most accepted" way to structure this is.

-> Generally I would suggest putting all of the library code in a subdirectory under src/.

pkg
  src
    Makevars
    R_pkg_specific_code.cc
      vender-code/
        vender-src.cc

See https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Compiling-in-sub_002ddirectories for how to setup your Makevars to do this (assuming the vendor library has a suitable makefile you can call).

See this pkg as example: https://github.com/cran/Rcsdp/tree/master/src



The R-extensions section on Makevars is a must read:
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-Makevars



I have read this article several times, but still feel confused about the conclusion of it, which is don’t use C++ to interface with R.
https://developer.r-project.org/Blog/public/2019/03/28/use-of-c---in-packages/index.html
-> The article is talking about writing C++ programs that make calls to R programs and explains the problems that can cause with the C++ code.
See also:
https://stat.ethz.ch/pipermail/r-devel/2019-March/077560.html

In my opinion (formed after re-reads, research, and even asking around)- the original negative r-project.org 1 article is wrong on some of its key technical points (for example: there is no C++ interface to R, article fails to emphasize Rcpp properly uses the C interface, article doesn't mention important history of .C versus .Call, and so on) and does not support a clear conclusion (let alone a conclusion of "don’t use C++ to interface with R"). I respect that the article is coming from a knowledgeable R authority, but feel the article has not been a good thing for the R community.

If the article had been carefully labeled as being only about "writing C++ programs that make calls to R" that would have helped. But it is hard to support that position as the original article starts with "About 20% packages from CRAN and BIOC repositories include some native code and more than a half of those include some code in C++". That commonly means R using C++ for extensions.

Perhaps the article should have been on a personal blog, or r-project.org 1 should become a formally refereed source (if one is going to treat it as authoritative).



ROOT is ...
https://root.cern/
A modular scientific software toolkit. It provides all the functionalities needed to deal with big data processing, statistical analysis, visualisation and storage. It is mainly written in C++ but integrated with other languages such as Python and R.



rr debuger (with reverse execution)
https://fitzgeraldnick.com/2015/11/02/back-to-the-futurre.html
https://rr-project.org/



# Video: Debugging compiled code in R with lldb (Jim Hester)
https://www.youtube.com/watch?v=R3-IMGyNJY4
# start lldb
R -d lldb

# clang commands
Ctrl-C to interrupt execution

r[estart] / c[continue]
up / down
bt
b
br list
br di
br en
br de

## All exceptions
break set -E c++

## Particular exception
break set -F std::range_error

# R funs

## Print value of object
Rf_PrintValue() / R_PV

## Inspect SEXP
R_inspect() / R_inspect3() # to limit the amount of recursion

## Print R Traceback from C
Rf_PrintValue(R_GetTraceback(0))

## Print Rcpp vector
Rf_PrintValue(vec.data)



Rf_PrintValue(). See also R_inspect(), and Rf_PrintValue(x->attrib) to view attributes of a SEXP.



# Video: Using gdb to debug a multi-module C code library loaded into R and called from R code
https://www.youtube.com/watch?v=SJdGxL0wrBA



debugit example package to practice debugging:
https://blog.davisvaughan.com/2019/04/05/debug-r-package-with-cpp/
https://github.com/DavisVaughan/debugit
devtools::install_github("DavisVaughan/debugit")



How to debug Rcpp code in a package
https://gist.github.com/alexg9010/1ab89ecb28c43f8bc217d2facca77f0a
 how_to_debug_rcpp.Rmd



http://dirk.eddelbuettel.com/papers/rcpp_uzuerich_2015_part5_packaging.pdf
Debugging is unfortunately platform-specific
- When the compiler isg++, the debugger isgdb
- When the compiler isclang++, the debugger islldb.
- I useg++more often (under Linux) so we’ll focus ongdb.
However,lldbis very simila


http://r-pkgs.had.co.nz/src.html#src-debugging



Calling a user-defined R function from C++ using Rcpp
https://stackoverflow.com/questions/21225662/calling-a-user-defined-r-function-from-c-using-rcpp?rq=1
#include <Rcpp.h>
using namespace Rcpp;

// [[Rcpp::export]]
SEXP mySuminC(){
  Environment myEnv = Environment::global_env();
  Function mySum = myEnv["mySum"];
  int x = myEnv["x"];
  int y = myEnv["y"];
  return mySum(Rcpp::Named("x", x), Rcpp::Named("y", y));
}
# (or, leave function return as int and use as<int> in place of wrap).



https://stackoverflow.com/questions/37665361/how-to-determine-the-class-of-object-stored-in-sexp-in-rcpp
https://stackoverflow.com/questions/24622918/initialize-a-variable-with-different-type-based-on-a-switch-statement
// the dispatch code

// [[Rcpp::export]]
SEXP GetDiag(SEXP pMat) {
  switch (TYPEOF(pMat)) {
  case INTSXP: return GetDiag_impl<IntegerMatrix>(pMat);
  case REALSXP: return GetDiag_impl<NumericMatrix>(pMat);
  <...>
  }


// attributes
// std::vector<std::string> nm = X.attributeNames();
// https://teuder.github.io/rcpp4everyone_en/202_attributes.html
// https://stackoverflow.com/questions/54693381/how-to-create-a-list-with-attributes-with-rcpp
// https://stackoverflow.com/questions/58965423/return-subset-of-a-given-sexp-without-knowing-the-actual-internal-data-type
// https://gallery.rcpp.org/articles/setting-object-attributes/



// Keep factors like shown here:
// https://stackoverflow.com/questions/58965423/return-subset-of-a-given-sexp-without-knowing-the-actual-internal-data-type?noredirect=1
// if (x.hasAttribute("levels")){
//   subset.attr("class") = x.attr("class");
//   subset.attr("levels") = x.attr("levels");
// }


https://stackoverflow.com/questions/47246200/how-to-slice-rcpp-numericvector-for-elements-2-to-101



Concatenate two vectors (sugar function may exist nowadays):
http://lists.r-forge.r-project.org/pipermail/rcpp-devel/2014-January/007025.html

template <const int RTYPE>
Vector<RTYPE> do_conc_(Vector<RTYPE> x, Vector<RTYPE> y){
  int nx=x.size(), n=x.size()+y.size(),i,j;
  Vector<RTYPE> out=no_init(n);
  for (i=0; i<nx; ++i){ out[ i ] = x[ i ];}
  for (j=i, i=0; j<n; ++j, ++i){ out[ j ] = y[i] ;}
  return out;
}

// [[Rcpp::export]]
SEXP conc( SEXP& XX_, SEXP& YY_){
  int type = TYPEOF(XX_) ;
  switch( type ){
  case INTSXP  : return do_conc_<INTSXP> ( XX_, YY_ ) ;
  case REALSXP : return do_conc_<REALSXP>( XX_, YY_ ) ;
  case STRSXP  : return do_conc_<STRSXP> ( XX_, YY_ ) ;
  case VECSXP  : return do_conc_<VECSXP> ( XX_, YY_ ) ;
  }
  return R_NilValue ;
}



    // Possible options to print values in Rcpp:
    // https://stackoverflow.com/questions/28152406/print-integer-vector-from-rcpp-function
    // https://stackoverflow.com/questions/35982284/how-to-print-an-r-object-to-stderr-in-rcpp
    // https://stackoverflow.com/questions/51169994/how-to-print-raw-values-in-rcpp
    // 1. Rf_PrintValue(a);
    // 2. Rcpp::print(a); // wraps the Rf_PrintValue
    // 3. Rcpp::Rcout << a << std::endl; // or Rcpp::Rcerr for output on the stderr device
    // SEXP a = Rcpp::wrap(x);
    // print(a);
    //
    // // Functions to print R objects
    // Rcpp::Rcout << "Debug output:" << std::endl;
    // Rf_PrintValue(x);
    // print(SEXPREC *)
    // print(y);
    // // R_PV(x);  // R API function (not from Rcpp). Calls Rf_PrintValue internally?
    // // R_PV was not declared in this scope
    // Rcpp::Rcout << "End of debug output" << std::endl;




Ideas to redirect R outpout from Rgui to gdb via R callback functions
https://stackoverflow.com/questions/59236579/gdb-prints-output-stdout-to-rgui-console-instead-of-gdb-console-on-windows-whe
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Setting-R-callbacks
https://opensourceforu.com/2012/02/function-pointers-and-callbacks-in-c-an-odyssey/
> These callbacks should never be changed in a running R session (and hence cannot be called from an extension package).
-> see the source code:
https://github.com/wch/r-source/blob/9b732e3338ed0f618ba18d136a68ca68733de3ef/src/unix/system.txt
https://github.com/wch/r-source/blob/9b732e3338ed0f618ba18d136a68ca68733de3ef/src/unix/system.c
https://github.com/wch/r-source/blob/06185bd431beb9b6e0883193273d1779ef2c6a2c/src/include/Rinterface.h#L136
https://fossies.org/dox/R-3.6.1/sys-std_8c.html
https://stat.ethz.ch/pipermail/r-devel/2011-May/060929.html   ->   cerr and cout not working calling c++ from R
!!! https://stackoverflow.com/questions/51242993/difference-between-r-3-4-4-and-r-3-5-1-in-r-api-for-c
void R_WriteConsole(const char *buf, int len) {if (ptr_R_WriteConsole) ptr_R_WriteConsole(buf, len); else ptr_R_WriteConsoleEx(buf, len, 0); }
void R_WriteConsoleEx(const char *buf, int len, int otype) {if (ptr_R_WriteConsole) ptr_R_WriteConsole(buf, len); else ptr_R_WriteConsoleEx(buf, len, otype); }
...
ptr_R_WriteConsole = Rstd_WriteConsole;
...
src/include/Rinterface.h:
  extern void (*ptr_R_WriteConsole)(const char *, int);
  extern void (*ptr_R_WriteConsoleEx)(const char *, int, int);
-------
Function: void R_WriteConsole (const char buf, int buflen)
Function: void R_WriteConsoleEx (const char buf, int buflen, int otype)
Function: void R_ResetConsole ()
Function: void R_FlushConsole ()
Function: void R_ClearErrConsole ()

    These functions interact with a console.

    R_ReadConsole prints the given prompt at the console and then does a fgets(3)–like operation, transferring up to buflen characters into the buffer buf. The last two bytes should be set to ‘“\n\0”’ to preserve sanity. If hist is non-zero, then the line should be added to any command history which is being maintained. The return value is 0 is no input is available and >0 otherwise.

    R_WriteConsoleEx writes the given buffer to the console, otype specifies the output type (regular output or warning/error). Call to R_WriteConsole(buf, buflen) is equivalent to R_WriteConsoleEx(buf, buflen, 0). To ensure backward compatibility of the callbacks, ptr_R_WriteConsoleEx is used only if ptr_R_WriteConsole is set to NULL. To ensure that stdout() and stderr() connections point to the console, set the corresponding files to NULL via

          R_Outputfile = NULL;
          R_Consolefile = NULL;

    R_ResetConsole is called when the system is reset after an error. R_FlushConsole is called to flush any pending output to the system console. R_ClearerrConsole clears any errors associated with reading from the console.

otype determines whether the content is
 *  regular output (0) or error/warning (1) and can be used by GUIs
 *  to distinguish those.

>>>>>>>> To implement a callback this may work (taken from Rgui):
https://github.com/wch/r-source/blob/5a156a0865362bb8381dcd69ac335f5174a4f60c/src/gnuwin32/system.c
// Global variables
static void (*TrueWriteConsole) (const char *, int);
static void (*TrueWriteConsoleEx) (const char *, int, int);
void R_WriteConsole(const char *buf, int len)
{
    R_ProcessEvents();
    if (TrueWriteConsole) TrueWriteConsole(buf, len);
    else TrueWriteConsoleEx(buf, len, 0);
}
void R_WriteConsoleEx(const char *buf, int len, int otype)
{
    R_ProcessEvents();
    if (TrueWriteConsole) TrueWriteConsole(buf, len);
    else TrueWriteConsoleEx(buf, len, otype);
}
    TrueWriteConsole = Rp->WriteConsole;   // Rstart Rp
    TrueWriteConsoleEx = Rp->WriteConsoleEx;
// https://github.com/wch/r-source/blob/06185bd431beb9b6e0883193273d1779ef2c6a2c/src/include/R_ext/RStartup.h#L97
structRstart:
   blah2 WriteConsole;  // typedef void (*blah2) (const char *, int);
   blah7 WriteConsoleEx; /* used only if WriteConsole is NULL */  // typedef void (*blah7) (const char *, int, int);
Rp->ReadConsole = myReadConsole;
Rp->WriteConsole = myWriteConsole;
static void myWriteConsole(const char *buf, int len)
{
    printf("%s", buf);
}
R_SetParams(Rp);
void R_SetWin32(Rstart Rp)  // https://github.com/wch/r-source/blob/7321904bbc57f59362c1f5ff8268f3702547e7e1/src/gnuwin32/system.c#L691
    TrueWriteConsole = Rp->WriteConsole;
    TrueWriteConsoleEx = Rp->WriteConsoleEx;


Rgui > break into debugger

Code see:
https://github.com/wch/r-source/blob/5a156a0865362bb8381dcd69ac335f5174a4f60c/src/gnuwin32/rui.c

void breaktodebugger(void)
{
    asm("int $3");
}

static Rboolean isdebuggerpresent(void)
{
    typedef BOOL (*R_CheckDebugger)(void);
    R_CheckDebugger entry;
    /* XP or later */
    entry =
	(R_CheckDebugger) GetProcAddress((HMODULE)GetModuleHandle("KERNEL32"),
					 "IsDebuggerPresent");
    if (entry == NULL) return(FALSE);
    else return (Rboolean) entry();
}

setconsoleoptions(... int buffered...):
consolebuffered = buffered;
https://github.com/wch/r-source/blob/5ffefa197f4b644fd85730fc795be964726a738a/src/gnuwin32/console.c#L1971



Rterm on Windows does support breakpoints and print gdb and R output to the same console!
But: It may be deprecated:
https://stackoverflow.com/questions/3412911/r-exe-rcmd-exe-rscript-exe-and-rterm-exe-whats-the-difference

Background why Ctrl+C works in Rterm (to break into debugger):
R.c:main:58 calls rcmdfn which calls stdlib.h:system() in rcmdfn.c:500 with cmd = "Rterm..."
rcmdfn.c:178 says: /* R.exe should ignore Ctrl-C, and let Rterm.exe handle it */
and calls:
SetConsoleCtrlHandler(NULL, TRUE); // included in Rfe.c:21 via windows.h
-> https://docs.microsoft.com/en-us/windows/console/setconsolectrlhandler
> Each console process has its own list of application-defined HandlerRoutine functions that handle CTRL+C and CTRL+BREAK signals.
< (...) The SetConsoleMode function can disable the ENABLE_PROCESSED_INPUT mode for a console's input buffer,
>       so CTRL+C is reported as keyboard input rather than as a signal. (...)
>       # https://docs.microsoft.com/en-us/windows/console/setconsolemode
> Calling SetConsoleCtrlHandler with the NULL and TRUE arguments causes the calling process to ignore CTRL+C signals.
> This attribute is inherited by child processes, but it can be enabled or disabled by any process without affecting existing processes.
> (...) If a console process is being debugged and CTRL+C signals have not been disabled, the system generates a DBG_CONTROL_C exception.
> This exception is raised only for the benefit of the debugger, and an application should never use an exception handler to deal with it.
> If the debugger handles the exception, an application will not notice the CTRL+C, with one exception: alertable waits will terminate.
> If the debugger passes the exception on unhandled, CTRL+C is passed to the console process and treated as a signal, as previously discussed.
> A console process can use the GenerateConsoleCtrlEvent function to send a CTRL+C or CTRL+BREAK signal to a console process group.
> # https://docs.microsoft.com/en-us/windows/console/generateconsolectrlevent
psignal.c:hwIntrHandler:135: Hardware interrupt handler for CTLR_C...
          case CTRL_C_EVENT :
          case CTRL_BREAK_EVENT :
              raise(SIGBREAK);
psignal.c:downhill_Signal_Init:165 sets the Hardware interrupt handler !!! Called by:
psignal.c:sigaction:190 -> called by?
psignal.c:signal:237 -> called in rterm.c:AppMain:126 -> signal(SIGBREAK, my_onintr); // to set the UserBreak = 1
psignal.c:sigpending:312
psignal.c:sigprocmask:337


Readme.rterm:
https://github.com/wch/r-source/blob/06185bd431beb9b6e0883193273d1779ef2c6a2c/src/gnuwin32/README.Rterm
Its purpose is to provide something similar to the main Unix interface
for R, including a batch capability.  The command-line arguments are
the same as those for RGui.exe and are documented in `An Introduction
to R' in the section `Invoking R under Windows' (--mdi and --sdi are
accepted but do nothing).  Batch use is covered in the rw-FAQ Q2.10.
In interactive use the interpreter can be interrupted by Ctrl-C or
Ctrl-Break.
...



INT 3 debugtrap (breakpoint):
https://stackoverflow.com/questions/173618/is-there-a-portable-equivalent-to-debugbreak-debugbreak
https://github.com/nemequ/portable-snippets/tree/master/debug-trap
https://github.com/nemequ/portable-snippets/
    __builtin_debugtrap for some versions of clang (identified with __has_builtin(__builtin_debugtrap))
    On MSVC and Intel C/C++ Compiler: __debugbreak
    For ARM C/C++ Compiler: __breakpoint(42)
    For x86/x86_64, assembly: int $03
    For ARM Thumb, assembly: .inst 0xde01
    For ARM AArch64, assembly: .inst 0xd4200000
    For other ARM, assembly: .inst 0xe7f001f0
    For Alpha, assembly: bpt
    For non-hosted C with GCC (or something which masquerades as it), __builtin_trap
    Otherwise, include signal.h and
        If defined(SIGTRAP) (i.e., POSIX), raise(SIGTRAP)
        Otherwise, raise(SIGABRT)

!!! The int 3 entry point is KiBreakpointTrap and int 2d entry is KiDebugServiceTrap
https://github.com/Zer0Mem0ry/ntoskrnl/blob/master/Ke/amd64/trap.asm
special look for line 1346 - inc qword ptr MfRip[rsp] this is where return address is synthetic incremented by +1. look also to line 340 - where in case int 3 exception address is decremented by -1 (to point exactly to int 3). and study how KiDispatchException worked. how Rip manipulated (line 497, etc).

DebugService Control types (eax register for int 2d):
https://doxygen.reactos.org/d1/dee/ndk_2kdtypes_8h_source.html#l00050
 #define BREAKPOINT_BREAK                    0
 #define BREAKPOINT_PRINT                    1
 #define BREAKPOINT_PROMPT                   2
 #define BREAKPOINT_LOAD_SYMBOLS             3
 #define BREAKPOINT_UNLOAD_SYMBOLS           4
 #define BREAKPOINT_COMMAND_STRING           5
the eax must be initialized before call int 2dh - the windows handle int 2dh differently, based on eax value. now known 5 values - say in case BREAKPOINT_PROMPT (2) windows generate exception breakpoint, when with 1,3,4 - auto handle it (in KdpStub if no kernel debugger attached). so set some correct value of eax is mandatory. after int 2dh windows synthetic increment rip (eip) by 1. as result we return (say after BREAKPOINT_PRINT (1)) exactly to ret - int 3 (1 byte) will be skipped. – RbMm Sep 3 '18 at 21:46


Windows Kernel API source code (not from Microsoft):
https://github.com/Zer0Mem0ry/ntoskrnl


Hardware interrupt handler in R (SIGBREAK):
https://github.com/wch/r-source/blob/5a156a0865362bb8381dcd69ac335f5174a4f60c/src/gnuwin32/psignal.c
https://github.com/wch/r-source/blob/5a156a0865362bb8381dcd69ac335f5174a4f60c/src/gnuwin32/fixed/h/psignal.h
https://github.com/wch/r-source/blob/5a156a0865362bb8381dcd69ac335f5174a4f60c/src/gnuwin32/psignal.c#L165
downhill_Signal_Init() -> SetConsoleCtrlHandler (hwIntrHandler, TRUE)
https://docs.microsoft.com/en-us/windows/console/setconsolectrlhandler
-> Each console process has its own list of application-defined HandlerRoutine functions that handle CTRL+C and CTRL+BREAK signals.

SetStdHandle(STD_ERROR_HANDLE, hOUT);

Windows documentation:
https://docs.microsoft.com/en-us/windows/


!!! How Windows 64 bit handles int 3:
http://x86asm.net/articles/debugging-in-amd64-64-bit-mode-in-theory/index.html#Software-Breakpoints
https://stackoverflow.com/questions/42053067/catching-stacktraces-with-debug-register-breakpoints-and-vectored-exception-hand

https://github.com/wch/r-source/blob/321ca345988101e5bb392427baf03a90d6ca2a46/src/main/main.c
/* Main Loop: It is assumed that at this point that operating system */
/* specific tasks (dialog window creation etc) have been performed. */
/* We can now print a greeting, run the .First function and then enter */
/* the read-eval-print loop. */
static RETSIGTYPE handleInterrupt(int dummy)
{
    R_interrupts_pending = 1;
    signal(SIGINT, handleInterrupt);
}



Fetch R-devel source code:
https://cran.r-project.org/doc/manuals/r-release/R-admin.html#Getting-and-unpacking-the-sources
https://cran.r-project.org/bin/windows/Rtools/
svn checkout https://svn.R-project.org/R/trunk <target sub folder>

-> see file "INSTALL" in the root folder for a how-to-build description

The Subversion repository does not contain the current sources for the recommended packages,
which can be obtained by rsync or downloaded from CRAN. To use rsync to install the appropriate sources for the recommended packages,
run ./tools/rsync-recommended from the top-level directory of the R sources.

You can use a separate build directory to separate source from build files:
https://bookdown.org/lionel/contributing/building-r.html#sec:build
# mkdir build
# cd build
# ../tools/rsync-recommended
# ../configure
# make

# Show all supported config options:
./configure --help

./configure:
configure: error: Some of the recommended packages are missing
  Use --without-recommended-packages if this was intentional

Solution:
cd tools
./rsync-recommended
cd ..
./configure
# or simply: make rsync-recommended


TODO:
make clean
make
# R with debugging information: https://cran.r-project.org/bin/windows/base/rw-FAQ.html#How-do-I-debug-code-that-I-have-compiled-and-dyn_002eload_002ded_003f
# make DEBUG=T  # do a make clean first since a recompile is avoid by make if files are unchanged
# make all recommended vignettes
# make manuals
make check
make check-recommended
make check-devel
make check-all
-> see file tests/README
-> fails are written to .Rout.fail file in the folder being checked

? (set ‘CFLAGS’ etc in config.site before configuring, or edit files Makeconf and etc/Makeconf between running configure and make)

Qt Creator generic project for R:
https://doc.qt.io/qtcreator/creator-project-generic.html#importing-a-generic-project
https://shinnok.com/rants/2011/07/18/using-the-qt-creator-ide-with-non-qt-projects/
- Import existing project
- add all project files to .files file (initially done by Qt Creator). Manual update:
  git ls-files "*.cpp" "*.h" > MyProject.files


Markdown extensions:
https://rapporter.github.io/pander/


OS X
User Makevars file is in home/.R
Makeconf is in /Library/Frameworks/R.framework/Versions/3.3/Resources/etc
# etc/Makeconf.  Generated from Makeconf.in by configure.
# CFLAGS = -Wall -mtune=core2 -g -O2 $(LTO)
CFLAGS = -Wall -mtune=core2 -g -O0 $(LTO)
# CXXFLAGS = -Wall -mtune=core2 -g -O2 $(LTO)
CXXFLAGS = -Wall -mtune=core2 -g -O0 $(LTO)
# CXX1XFLAGS = -Wall -mtune=core2 -g -O2
CXX1XFLAGS = -Wall -mtune=core2 -g -O0




lldb usage:
https://lldb.llvm.org/use/tutorial.html

A process, by default, will share the lldb terminal with the inferior process.
When in this mode, much like when debugging with gdb, when the process is running anything you type will go to the STDIN of the inferior process.

To interrupt your inferior program, type CTRL+C.

Questions:

(lldb) frame variable  # says
(Rcpp::NumericVector) nv = <no location, value may have been optimized out>
# why? -O0 makes no diff to -O2! -fPIC  or  -mtune=core2?
# PIC = position independent code (eg. only relative jumps)
# clang++ -std=c++11 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o CppDebugHelper.so R_and_Rcpp_data_types_doc.o RcppExports.o _playground_.o break_to_debugger.o dbg_attributes.o dbg_get.o dbg_ls.o dbg_print.o dbg_str.o dbg_subset.o dbg_syscalls.o dbg_table.o debug_example_rcpp.o utils.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation

Intellisense for and calls of dbg_* functions do not work. Why?
(lldb) call dbg_print(nv)
error: Couldn't lookup symbols:
  __Z9dbg_printN4Rcpp6VectorILi14ET_EE
# call dbg_ls() does work! Also:
# call dbg_print("a")
# call dbg_print(nv->data)