NEWS.md

iNZightTools 2.0.1

• Fix issue where NULL values passed to interpolate() triggering an error

iNZightTools 2.0.0

Breaking changes

• major code rework for all data-wrangling and variable manipulation functions with tidyverse libraries and development conventions
• renamed most data-wrangling and variable manipulation functions such that their functionality are more name-intuitive (see below)

Minor changes

• Default code of generated data from all data-wrangling and variable manipulation functions now uses native R pipe |>. An option is added to use the magrittr pipe %>% instead.
• The code() accessor function now returns a string with class inzcode that has a print method which formats the code nicely.

New features

• The following functions are reworked and added new features:
  • renamed aggregatedt() to aggregate_dt() and aggregate_dt() is now a wrapper of aggregate_data() with date component extraction and aggregation functionality built in, and added an option to allow aggregation with additional variables
  • combined unite() and combineCatVars() into combine_vars() and added options to remove empty factor combinations and turn missing values into an explicit factor level
  • combined reshape_data() and stackVars() into reshape_data() and reworked the arguments
  • renamed rankVars() to rank_vars() and included more ranking methods (integer ranking by density and percentage ranking)
  • renamed reorderLevels() to reorder_levels and included more auto-reordering methods (by first appearance and numeric order)
  • added an option to specify time zone in convert_to_datetime()
  • renamed joindata() to join_data(), included right-join method, and reworked its arguments
• pull %||% operator from 'iNZight'

Other changes

• The following functions have changed names and with renamed/reworked arguments (see code documentation for more details)
  • renamed aggregateData() to aggregate_data()
  • renamed appendrows() to append_rows()
  • renamed collapseLevels() to collapse_cat()
  • renamed convertToCat() to convert_to_cat()
  • renamed createNewVar() to create_vars()
  • renamed deleteVars() to delete_vars()
  • renamed extract_part() to extract_dt_comp()
  • renamed filterLevels() to filter_cat()
  • renamed filterNumeric() to filter_num()
  • renamed filterRandom() to random_sample()
  • renamed filterRows() to remove_rows()
  • renamed missingToCat() to missing_to_cat()
  • renamed renameLevels() to rename_levels()
  • renamed renameVars() to rename_vars()
  • renamed selectVars() to select_vars()
  • renamed separate() to separate_var()
  • renamed sortVars() to sort_vars()
  • renamed standardizeVars() to standardize_vars()
  • renamed transformVar() to transform_vars()
• removed countMissing()

iNZightTools 1.13.4

• read_text() function handles all invalid characters (as smart_read() does)
• read_text() applies string to factor conversion

iNZightTools 1.13.3

• read_text() function replaces spaces with underscores in column names

iNZightTools 1.13.2

• add global options to set/override default comment character (this will allow Lite to change the default without changing the package's default behaviour; default set as # at load time)

iNZightTools 1.13.0

New features

Data dictionaries

Users can now import a data dictionary and apply it to a dataset. This will apply text labels to numerically coded variables and set labels for variables with human-friendly names.

Linked data

New functions to load linked datasets from a .inzlnk file, where the file linkage is specified. Optionally users can include a data dictionary in this file, which will also be applied to the data.

Other changes

• relocate survey specification reading files to new package surveyspec
• add survival analysis as an option for fitModel via family = "cox"
• initial implementation of (basic) database connectivity - this is in early alpha stage
• various bug fixes/changes
• fix some issues occuring when column names have spaces (in the CSV) (#200)

iNZightTools 1.12.3

• allow 'readr' to use automatic delimiter guessing (csv files) instead of forcing , (#187)

• fix Windows bug in url_to_temp() adding extra lines, causing importing files with more than one line of comments to fail delimiter-guessing

• fix bug where extension guess failed if file extension was not lower case (e.g., file.CSV)

• fix bug in converting times with missing seconds (e.g., '08:30')

• read_dictionary() and apply_dictionary() functions to import a data dictionary from a rectangular data file and apply it to a dataset

iNZightTools 1.12.2

• add new function print_code() to extract, tidy, and print code attached to an object
• allow more than one character as 'sep' argument to combineCatVars()

iNZightTools 1.12.1

• fix bug in smart_read() where parsing column types (numeric -> categorical) failed if NAs in column
• fix test failing in new version of survey (2.4)

iNZightTools 1.12.0

• add new support for metadata formats:
  • multiple-response variables (using @multi)
  • numeric na codes (using @numeric x na=99)
• allow spaces in new factor level names (renameLevels())
• add additional time formats for extract_part()
• add as_survey() method for inzsvyspec objects (#178)
• drops support for R 3.6 due to lack of RCurl binaries
• prevent showing col types with read_text()
• pass lazy = FALSE to readr::read_delim() to prevent lazy loading of data
• handle missing 'reptype' argument in survey specification file (defaults to 'BRR', the survey::svrepdesign() default)

iNZightTools 1.11.4

• fix bug in aggregateData() if summary_vars is not specified and there are factor columns in the dataset not showing up in vars

iNZightTools 1.11.3

• fix form_class_intervals to work with survey designs (count not yet working)
• remove accents/special chars from factor levels when parsing metadata (due to Windows encoding issues)

iNZightTools 1.11.2

• prevents tests from failing during CRAN checks when resources unavailable

iNZightTools 1.11.1

• update survey_IQR (and some tests) to work with 'survey' 4.1 (new svyquantile function)

iNZightTools 1.11

• use srvyr package to handle survey designs using dplyr-like syntax
  • aggregateData(), filterNumeric(), filterLevels(), filterRows(), ...
• remove redundant code from filterLevels() to make it more concise (using droplevels())
• add method to convert survey design-like objects to iNZight's inzsvyspec format
• suppress printing of dataset when joining data
• fix bug where special characters in levels during collapseLevels() cause error
• add new form_class_intervals() function
• update import_survey() to handle URL as the data argument
• fix bug in reorderLevels(..., freq = TRUE) (#165, @tmelliott)

iNZightTools 1.10

• refactor aggregateData function to improve flexibility
• new import_survey() function for importing a survey design (and data) from a specification file (in TOML format)

iNZightTools 1.9.1

• fix bug in reading delim files where, if first 1000+ rows of a column were NA, the column was read as logical instead of character (this is a fixed behaviour of readr, so cannot be overridden)

iNZightTools 1.9.0

Minor version bump to align with release of iNZight 4.0.

• add functions for reading survey design from a file (and applying to dataset)
• add support for reading JSON files
• add new selectVars function
• add negative binomial as an option for family in fitModel
• add %notin% operator to more easily/readably check ! x %in% y
• fitModel can now fit Cox PH models

---

iNZightTools 1.8.6

• ensure non-numeric values are returned as factors by read_text()

iNZightTools 1.8.5

• increase nmax for previews to 100 (from 10)
• use forcats::fct_cross() with optional argument keep_empty

iNZightTools 1.8.4

• fix handling of special characters in column names (spaces replaces with underscores, all others replaced with a period (.))
• create make_names() function (previously from 'iNZight') to create unique new variable names for columns in a data.frame

iNZightTools 1.8.3

• new add_suffix() function to smartly add suffixes to object names and append counter if suffix is already present
• edit package description to meet CRAN standards

iNZightTools 1.8.2

• fix issues from CRAN submission

iNZightTools 1.8.1

• new read_text() function to read from text string/clipboard
• smart_read() now supports URLs by downloading to a temporary file, with the same name
• Extract from datetime no longer includes space in 2010M01, etc. so it works correctly with iNZightTS
• Reshape (wide to long) returns a factor to work with the rest of iNZight

iNZightTools 1.8

Release date: 11 November 2019

• ensure column_types argument is respected
• if no col types specified, pass col_types = cols() to suppress the col spec messages
• new load_rda() function which loads all data frames in an rda file into a list
• new save_rda()) function allows saving of a data set to a file, optionally with a different name (i.e., the actual object name can be changed)
• smart_read() now handles RDS files
• add attribute to preview excel containing names of available sheets

iNZightTools 1.7.4

Release date: 2 September 2019

• fix bug where collapsing a "numeric" factor was giving an error
• fix bug in reordering same factor twice

iNZightTools 1.7.3

Release date: 26 August 2019

• fixes bug where factor order specified by metadata wasn't being respected
• fix bug preventing first row of metadata comments to be read if description was missing

iNZightTools 1.7.2

Release date: 15 July 2019

• various bug fixes

iNZightTools 1.7.1

Release date: 30 April 2019

• add appveyor ci
• add code tidying functionality
• import SAS files (.sas7bdat and .xpt)
• validate datasets
• join data using *_join()
• join data by rows
• reshape data from long to wide, and vice versa
• convert characters to factors (this is how iNZight does things)
• respect numerical order in factor conversions

iNZightTools 1.6.3

Release date: 15 November 2018

• [fix] prefix survey:: namespace to function calls

iNZightTools 1.6.2

Release date: 10 September 2018

• fix a bug in reading csv files with spaces in the header

iNZightTools 1.6.1

Release date: 04 September 2018

• fix bug in reading metadata where non-meta comments would cancel read

iNZightTools 1.6

Release date: 14 August 2018

Breaking Changes

• data import is now performed by the smart_read() function

Minor Changes and Fixes

• new function to generate R variable names (for code writing)
• various bug fixes

Version 1.5

Release date: 23 January 2018

Tidyverse and Code Writing

• data manipulation functions rewritten using tidyverse functions/workflow
• functions written such that the code is generated and evaluated, allowing the code history to be obtained

Patches

Patch 1.5.1 - 21/02/2018

• fix tiny bug in encoding default that prevented non-UTF-8 files from reading

Patch 1.5.2 - 22/02/2018

• fix a bug where timezone NA on macs prevented reading data

Patch 1.5.3 - 12/03/2018

• fix a bug in csv reading on macos

---

Version 1.4

Release date: 25 August 2017

• Code history features: interpolate code
• Re-document package; pass CRAN checks
• start converting data modification functions to tidyverse

Patches

Version 1.4-1, 02/10/2017

• fixes to pass CRAN checks

---

Version 1.3

Release date: 23 March 2017

• NEW import data function, as well as support for metadata at the top of text files
• Various bug fixes

---

Version 1.2

Release date: 5 September, 2016

• New device function uses the cairo device on Linux

Patch 1.2.1 - 3 February, 2017

• Only use the Acinonyx package on Mac (crashes R on Linux)

Patch 1.2.2 - 18 February, 2017

• Stop using type = "nbcairo" as macOS fallback in newdevice()

---

Version 1.1

• Directly access svydesign function. Temporary fix until package gets a more significant revamp.

---

Version 1.0

Initial release.