From e6918187568dbd01842d8d1d2c808ce16a894239 Mon Sep 17 00:00:00 2001
From: Daniel Baumann <daniel.baumann@progress-linux.org>
Date: Sun, 21 Apr 2024 13:54:28 +0200
Subject: Adding upstream version 18.2.2.

Signed-off-by: Daniel Baumann <daniel.baumann@progress-linux.org>
---
 src/arrow/r/vignettes/install.Rmd | 448 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 448 insertions(+)
 create mode 100644 src/arrow/r/vignettes/install.Rmd

(limited to 'src/arrow/r/vignettes/install.Rmd')

diff --git a/src/arrow/r/vignettes/install.Rmd b/src/arrow/r/vignettes/install.Rmd
new file mode 100644
index 000000000..5bd76a371
--- /dev/null
+++ b/src/arrow/r/vignettes/install.Rmd
@@ -0,0 +1,448 @@
+---
+title: "Installing the Arrow Package on Linux"
+output: rmarkdown::html_vignette
+vignette: >
+  %\VignetteIndexEntry{Installing the Arrow Package on Linux}
+  %\VignetteEngine{knitr::rmarkdown}
+  %\VignetteEncoding{UTF-8}
+---
+
+On macOS and Windows, when you `install.packages("arrow")`,
+you get a binary package that contains Arrow’s C++ dependencies along with it.
+On Linux, `install.packages()` retrieves a source package that has to be compiled locally,
+and C++ dependencies need to be resolved as well.
+Generally for R packages with C++ dependencies,
+this requires either installing system packages, which you may not have privileges to do,
+or building the C++ dependencies separately,
+which introduces all sorts of additional ways for things to go wrong.
+
+Our goal is to make `install.packages("arrow")` "just work" for as many Linux distributions,
+versions, and configurations as possible.
+This document describes how it works and the options for fine-tuning Linux installation.
+The intended audience for this document is `arrow` R package users on Linux, not developers.
+If you're contributing to the Arrow project, see `vignette("developing", package = "arrow") for guidance on setting up your development environment.
+
+Note also that if you use `conda` to manage your R environment, this document does not apply.
+You can `conda install -c conda-forge --strict-channel-priority r-arrow` and you'll get the latest official
+release of the R package along with any C++ dependencies.
+
+> Having trouble installing `arrow`? See the "Troubleshooting" section below.
+
+# Installation basics
+
+Install the latest release of `arrow` from CRAN with
+
+```r
+install.packages("arrow")
+```
+
+Daily development builds, which are not official releases,
+can be installed from the Ursa Labs repository:
+
+```r
+install.packages("arrow", repos = "https://arrow-r-nightly.s3.amazonaws.com")
+```
+
+or for conda users via:
+
+```
+conda install -c arrow-nightlies -c conda-forge --strict-channel-priority r-arrow
+```
+
+You can also install the R package from a git checkout:
+
+```shell
+git clone https://github.com/apache/arrow
+cd arrow/r
+R CMD INSTALL .
+```
+
+If you don't already have the Arrow C++ libraries on your system,
+when installing the R package from source, it will also download and build
+the Arrow C++ libraries for you. To speed installation up, you can set
+
+```shell
+export LIBARROW_BINARY=true
+```
+
+to look for C++ binaries prebuilt for your Linux distribution/version.
+Alternatively, you can set
+
+```shell
+export LIBARROW_MINIMAL=false
+```
+
+to build the Arrow libraries from source with optional features such as compression libraries
+enabled. This will increase the build time but provides many useful features.
+Prebuilt binaries are built with this flag enabled, so you get the full
+functionality by using them as well.
+
+Both of these variables are also set this way if you have the `NOT_CRAN=true`
+environment variable set.
+
+## Helper function: install_arrow()
+
+If you already have `arrow` installed and want to upgrade to a different version,
+install a development build, or try to reinstall and fix issues with Linux
+C++ binaries, you can call `install_arrow()`.
+`install_arrow()` provides some convenience wrappers around the various
+environment variables described below.
+This function is part of the `arrow` package,
+and it is also available as a standalone script, so you can
+access it for convenience without first installing the package:
+
+```r
+source("https://raw.githubusercontent.com/apache/arrow/master/r/R/install-arrow.R")
+```
+
+`install_arrow()` will install from CRAN,
+while `install_arrow(nightly = TRUE)` will give you a development build.
+`install_arrow()` does not require environment variables to be set in order to
+satisfy C++ dependencies.
+
+> Note that, unlike packages like `tensorflow`, `blogdown`, and others that require external dependencies, you do not need to run `install_arrow()` after a successful `arrow` installation.
+
+## Offline installation
+
+The `install-arrow.R` file also includes the `create_package_with_all_dependencies()`
+function. Normally, when installing on a computer with internet access, the
+build process will download third-party dependencies as needed.
+This function provides a way to download them in advance.
+Doing so may be useful when installing Arrow on a computer without internet access.
+Note that Arrow _can_ be installed on a computer without internet access without doing this, but
+many useful features will be disabled, as they depend on third-party components.
+More precisely, `arrow::arrow_info()$capabilities()` will be `FALSE` for every
+capability.
+One approach to add more capabilities in an offline install is to prepare a
+package with pre-downloaded dependencies. The
+`create_package_with_all_dependencies()` function does this preparation.
+
+If you're using binary packages you shouldn't need to follow these steps. You
+should download the appropriate binary from your package repository, transfer
+that to the offline computer, and install that. Any OS can create the source
+bundle, but it cannot be installed on Windows. (Instead, use a standard
+Windows binary package.)
+
+Note if you're using RStudio Package Manager on Linux: If you still want to
+make a source bundle with this function, make sure to set the first repo in
+`options("repos")` to be a mirror that contains source packages (that is:
+something other than the RSPM binary mirror URLs).
+
+### Using a computer with internet access, pre-download the dependencies:
+* Install the `arrow` package _or_ run
+  `source("https://raw.githubusercontent.com/apache/arrow/master/r/R/install-arrow.R")`
+* Run `create_package_with_all_dependencies("my_arrow_pkg.tar.gz")`
+* Copy the newly created `my_arrow_pkg.tar.gz` to the computer without internet access
+
+### On the computer without internet access, install the prepared package:
+* Install the `arrow` package from the copied file
+  * `install.packages("my_arrow_pkg.tar.gz", dependencies = c("Depends", "Imports", "LinkingTo"))`
+  * This installation will build from source, so `cmake` must be available
+* Run `arrow_info()` to check installed capabilities
+
+#### Alternative, hands-on approach
+* Download the dependency files (`cpp/thirdparty/download_dependencies.sh` may be helpful)
+* Copy the directory of dependencies to the offline computer
+* Create the environment variable `ARROW_THIRDPARTY_DEPENDENCY_DIR` on the offline computer, pointing to the copied directory.
+* Install the `arrow` package as usual.
+
+## S3 support
+
+The `arrow` package allows you to work with data in AWS S3 or in other cloud
+storage system that emulate S3. However, support for working with S3 is not
+enabled in the default build, and it has additional system requirements. To
+enable it, set the environment variable `LIBARROW_MINIMAL=false` or
+`NOT_CRAN=true` to choose the full-featured build, or more selectively set
+`ARROW_S3=ON`. You also need the following system dependencies:
+
+* `gcc` >= 4.9 or `clang` >= 3.3; note that the default compiler on CentOS 7 is gcc 4.8.5, which is not sufficient
+* CURL: install `libcurl-devel` (rpm) or `libcurl4-openssl-dev` (deb)
+* OpenSSL >= 1.0.2: install `openssl-devel` (rpm) or `libssl-dev` (deb)
+
+The prebuilt C++ binaries come with S3 support enabled, so you will need to meet
+these system requirements in order to use them--the package will not install
+without them. If you're building everything from source, the install script
+will check for the presence of these dependencies and turn off S3 support in the
+build if the prerequisites are not met--installation will succeed but without
+S3 functionality. If afterwards you install the missing system requirements,
+you'll need to reinstall the package in order to enable S3 support.
+
+# How dependencies are resolved
+
+In order for the `arrow` R package to work, it needs the Arrow C++ library.
+There are a number of ways you can get it: a system package; a library you've
+built yourself outside of the context of installing the R package;
+or, if you don't already have it, the R package will attempt to resolve it
+automatically when it installs.
+
+If you are authorized to install system packages and you're installing a CRAN release,
+you may want to use the official Apache Arrow release packages corresponding to the R package version (though there are some drawbacks: see "Troubleshooting" below).
+See the [Arrow project installation page](https://arrow.apache.org/install/)
+to find pre-compiled binary packages for some common Linux distributions,
+including Debian, Ubuntu, and CentOS.
+You'll need to install `libparquet-dev` on Debian and Ubuntu, or `parquet-devel` on CentOS.
+This will also automatically install the Arrow C++ library as a dependency.
+
+When you install the `arrow` R package on Linux,
+it will first attempt to find the Arrow C++ libraries on your system using
+the `pkg-config` command.
+This will find either installed system packages or libraries you've built yourself.
+In order for `install.packages("arrow")` to work with these system packages,
+you'll need to install them before installing the R package.
+
+If no Arrow C++ libraries are found on the system,
+the R package installation script will next attempt to download
+prebuilt static Arrow C++ libraries
+that match your both your local operating system and `arrow` R package version.
+C++ binaries will only be retrieved if you have set the environment variable
+`LIBARROW_BINARY` or `NOT_CRAN`.
+If found, they will be downloaded and bundled when your R package compiles.
+For a list of supported distributions and versions,
+see the [arrow-r-nightly](https://github.com/ursa-labs/arrow-r-nightly/blob/master/README.md) project.
+
+If no C++ library binary is found, it will attempt to build it locally.
+First, it will also look to see if you are in
+a checkout of the `apache/arrow` git repository and thus have the C++ source there.
+Otherwise, it builds from the C++ files included in the package.
+Depending on your system, building Arrow C++ from source may be slow.
+
+For the specific mechanics of how all this works, see the R package `configure` script,
+which calls `tools/nixlibs.R`.
+
+If the C++ library is built from source, `inst/build_arrow_static.sh` is executed.
+This build script is also what is used to generate the prebuilt binaries.
+
+## How the package is installed - advanced
+
+This subsection contains information which is likely to be most relevant mostly
+to Arrow developers and is not necessary for Arrow users to install Arrow.
+
+There are a number of scripts that are triggered when `R CMD INSTALL .` is run.
+For Arrow users, these should all just work without configuration and pull in
+the most complete pieces (e.g. official binaries that we host).
+
+An overview of these scripts is shown below:
+
+* `configure` and `configure.win` - these scripts are triggered during
+`R CMD INSTALL .` on non-Windows and Windows platforms, respectively. They
+handle finding the Arrow library, setting up the build variables necessary, and
+writing the package Makevars file that is used to compile the C++ code in the R
+package.
+
+* `tools/nixlibs.R` - this script is sometimes called by `configure` on Linux
+(or on any non-windows OS with the environment variable
+`FORCE_BUNDLED_BUILD=true`). This sets up the build process for our bundled
+builds (which is the default on linux). The operative logic is at the end of
+the script, but it will do the following (and it will stop with the first one
+that succeeds and some of the steps are only checked if they are enabled via an
+environment variable):
+  * Check if there is an already built libarrow in `arrow/r/libarrow-{version}`,
+  use that to link against if it exists.
+  * Check if a binary is available from our hosted unofficial builds.
+  * Download the Arrow source and build the Arrow Library from source.
+  * `*** Proceed without C++` dependencies (this is an error and the package
+  will not work, but if you see this message you know the previous steps have
+  not succeeded/were not enabled)
+
+* `inst/build_arrow_static.sh` - called by `tools/nixlibs.R` when the Arrow
+library is being built.  It builds Arrow for a bundled, static build, and
+mirrors the steps described in the ["Arrow R Developer Guide" vignette](./developing.html)
+
+# Troubleshooting
+
+The intent is that `install.packages("arrow")` will just work and handle all C++
+dependencies, but depending on your system, you may have better results if you
+tune one of several parameters. Here are some known complications and ways to address them.
+
+## Package failed to build C++ dependencies
+
+If you see a message like
+
+```
+------------------------- NOTE ---------------------------
+There was an issue preparing the Arrow C++ libraries.
+See https://arrow.apache.org/docs/r/articles/install.html
+---------------------------------------------------------
+```
+
+in the output when the package fails to install,
+that means that installation failed to retrieve or build C++ libraries
+compatible with the current version of the R package.
+
+It is expected that C++ dependencies should be built successfully
+on all Linux distributions, so you should not see this message. If you do,
+please check the "Known installation issues" below to see if any apply.
+If none apply, set the environment variable `ARROW_R_DEV=TRUE`
+so that details on what failed are shown, and try installing again. Then,
+please [report an issue](https://issues.apache.org/jira/projects/ARROW/issues)
+and include the full verbose installation output.
+
+## Using system libraries
+
+If a system library or other installed Arrow is found but it doesn't match the R package version
+(for example, you have libarrow 1.0.0 on your system and are installing R package 2.0.0),
+it is likely that the R bindings will fail to compile.
+Because the Apache Arrow project is under active development,
+is it essential that versions of the C++ and R libraries match.
+When `install.packages("arrow")` has to download the C++ libraries,
+the install script ensures that you fetch the C++ libraries that correspond to your R package version.
+However, if you are using Arrow libraries already on your system, version match isn't guaranteed.
+
+To fix version mismatch, you can either update your system packages to match the R package version,
+or set the environment variable `ARROW_USE_PKG_CONFIG=FALSE`
+to tell the configure script not to look for system Arrow packages.
+(The latter is the default of `install_arrow()`.)
+System packages are available corresponding to all CRAN releases
+but not for nightly or dev versions, so depending on the R package version you're installing,
+system packages may not be an option.
+
+Note also that once you have a working R package installation based on system (shared) libraries,
+if you update your system Arrow, you'll need to reinstall the R package to match its version.
+Similarly, if you're using Arrow system libraries, running `update.packages()`
+after a new release of the `arrow` package will likely fail unless you first
+update the system packages.
+
+## Using prebuilt binaries
+
+If the R package finds and downloads a prebuilt binary of the C++ library,
+but then the `arrow` package can't be loaded, perhaps with "undefined symbols" errors,
+please [report an issue](https://issues.apache.org/jira/projects/ARROW/issues).
+This is likely a compiler mismatch and may be resolvable by setting some
+environment variables to instruct R to compile the packages to match the C++ library.
+
+A workaround would be to set the environment variable `LIBARROW_BINARY=FALSE`
+and retry installation: this value instructs the package to build the C++ library from source
+instead of downloading the prebuilt binary.
+That should guarantee that the compiler settings match.
+
+If a prebuilt binary wasn't found for your operating system but you think it should have been,
+check the logs for a message that says `*** Unable to identify current OS/version`,
+or a message that says `*** No C++ binaries found for` an invalid OS.
+If you see either, please [report an issue](https://issues.apache.org/jira/projects/ARROW/issues).
+You may also set the environment variable `ARROW_R_DEV=TRUE` for additional
+debug messages.
+
+A workaround would be to set the environment variable `LIBARROW_BINARY`
+to a `distribution-version` that exists in the Ursa Labs repository.
+Setting `LIBARROW_BINARY` is also an option when there's not an exact match
+for your OS but a similar version would work,
+such as if you're on `ubuntu-18.10` and there's only a binary for `ubuntu-18.04`.
+
+If that workaround works for you, and you believe that it should work for everyone else too,
+you may propose [adding an entry to this lookup table](https://github.com/ursa-labs/arrow-r-nightly/edit/master/linux/distro-map.csv).
+This table is checked during the installation process
+and tells the script to use binaries built on a different operating system/version
+because they're known to work.
+
+## Building C++ from source
+
+If building the C++ library from source fails, check the error message.
+(If you don't see an error message, only the `----- NOTE -----`,
+set the environment variable `ARROW_R_DEV=TRUE` to increase verbosity and retry installation.)
+The install script should work everywhere, so if the C++ library fails to compile,
+please [report an issue](https://issues.apache.org/jira/projects/ARROW/issues)
+so that we can improve the script.
+
+## Known installation issues
+
+* On CentOS, if you are using a more modern `devtoolset`, you may need to set
+the environment variables `CC` and `CXX` either in the shell or in R's `Makeconf`.
+For CentOS 7 and above, both the Arrow system packages and the C++ binaries
+for R are built with the default system compilers. If you want to use either of these
+and you have a `devtoolset` installed, set `CC=/usr/bin/gcc CXX=/usr/bin/g++`
+to use the system compilers instead of the `devtoolset`.
+Alternatively, if you want to build `arrow` with the newer `devtoolset` compilers,
+set both `ARROW_USE_PKG_CONFIG` and `LIBARROW_BINARY` to `false` so that
+you build the Arrow C++ from source using those compilers.
+Compiler mismatch between the arrow system libraries and the R
+package may cause R to segfault when `arrow` package functions are used.
+See discussions [here](https://issues.apache.org/jira/browse/ARROW-8586)
+and [here](https://issues.apache.org/jira/browse/ARROW-10780).
+
+* If you have multiple versions of `zstd` installed on your system,
+installation by building the C++ from source may fail with an undefined symbols
+error. Workarounds include (1) setting `LIBARROW_BINARY` to use a C++ binary; (2)
+setting `ARROW_WITH_ZSTD=OFF` to build without `zstd`; or (3) uninstalling
+the conflicting `zstd`.
+See discussion [here](https://issues.apache.org/jira/browse/ARROW-8556).
+
+## Summary of build environment variables
+
+Some features are optional when you build Arrow from source. With the exception of `ARROW_S3`, these are all `ON` by default in the bundled C++ build, but you can set them to `OFF` to disable them.
+
+* `ARROW_S3`: If set to `ON` S3 support will be built as long as the
+  dependencies are met; if they are not met, the build script will turn this `OFF`
+* `ARROW_JEMALLOC` for the `jemalloc` memory allocator
+* `ARROW_MIMALLOC` for the `mimalloc` memmory allocator
+* `ARROW_PARQUET`
+* `ARROW_DATASET`
+* `ARROW_JSON` for the JSON parsing library
+* `ARROW_WITH_RE2` for the RE2 regular expression library, used in some string compute functions
+* `ARROW_WITH_UTF8PROC` for the UTF8Proc string library, used in many other string compute functions
+* `ARROW_JSON` for JSON parsing
+* `ARROW_WITH_BROTLI`, `ARROW_WITH_BZ2`, `ARROW_WITH_LZ4`, `ARROW_WITH_SNAPPY`, `ARROW_WITH_ZLIB`, and `ARROW_WITH_ZSTD` for various compression algorithms
+
+
+There are a number of other variables that affect the `configure` script and the bundled build script.
+By default, these are all unset. All boolean variables are case-insensitive.
+
+* `ARROW_USE_PKG_CONFIG`: If set to `false`, the configure script
+  won't look for Arrow libraries on your system and instead will look to download/build them.
+  Use this if you have a version mismatch between installed system libraries
+  and the version of the R package you're installing.
+* `LIBARROW_BINARY`: If set to `true`, the script will try to download a binary
+  C++ library built for your operating system.
+  You may also set it to some other string,
+  a related "distro-version" that has binaries built that work for your OS.
+  If no binary is found, installation will fall back to building C++
+  dependencies from source.
+* `LIBARROW_BUILD`: If set to `false`, the build script
+  will not attempt to build the C++ from source. This means you will only get
+  a working `arrow` R package if a prebuilt binary is found.
+  Use this if you want to avoid compiling the C++ library, which may be slow
+  and resource-intensive, and ensure that you only use a prebuilt binary.
+* `LIBARROW_MINIMAL`: If set to `false`, the build script
+  will enable some optional features, including compression libraries, S3
+  support, and additional alternative memory allocators. This will increase the
+  source build time but results in a more fully functional library.
+* `NOT_CRAN`: If this variable is set to `true`, as the `devtools` package does,
+  the build script will set `LIBARROW_BINARY=true` and `LIBARROW_MINIMAL=false`
+  unless those environment variables are already set. This provides for a more
+  complete and fast installation experience for users who already have
+  `NOT_CRAN=true` as part of their workflow, without requiring additional
+  environment variables to be set.
+* `ARROW_R_DEV`: If set to `true`, more verbose messaging will be printed
+  in the build script. `arrow::install_arrow(verbose = TRUE)` sets this.
+  This variable also is needed if you're modifying C++
+  code in the package: see the developer guide vignette.
+* `LIBARROW_DEBUG_DIR`: If the C++ library building from source fails (`cmake`),
+  there may be messages telling you to check some log file in the build directory.
+  However, when the library is built during R package installation,
+  that location is in a temp directory that is already deleted.
+  To capture those logs, set this variable to an absolute (not relative) path
+  and the log files will be copied there.
+  The directory will be created if it does not exist.
+* `CMAKE`: When building the C++ library from source, you can specify a
+  `/path/to/cmake` to use a different version than whatever is found on the `$PATH`
+
+# Contributing
+
+As mentioned above, please [report an issue](https://issues.apache.org/jira/projects/ARROW/issues)
+if you encounter ways to improve this. If you find that your Linux distribution
+or version is not supported, we welcome the contribution of Docker images
+(hosted on Docker Hub) that we can use in our continuous integration. These
+Docker images should be minimal, containing only R and the dependencies it
+requires. (For reference, see the images that
+[R-hub](https://github.com/r-hub/rhub-linux-builders) uses.)
+
+You can test the `arrow` R package installation using the `docker-compose`
+setup included in the `apache/arrow` git repository. For example,
+
+```
+R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest docker-compose build r
+R_ORG=rhub R_IMAGE=ubuntu-gcc-release R_TAG=latest docker-compose run r
+```
+
+installs the `arrow` R package, including the C++ source build, on the
+[rhub/ubuntu-gcc-release](https://hub.docker.com/r/rhub/ubuntu-gcc-release)
+image.
-- 
cgit v1.2.3

