The following features are available in Busybox ash, so we don't need to
warn about them for runtime scripts:

- local keyword
- echo -n / -e
- String indexing

These warnings are new in shellcheck 0.7.2, which would otherwise fail
for various scripts.

Resolves #2362

All our targets use the OpenWrt device abstraction. Since commit
6ba58c9b17c90e41b521d796ab76e5723ee017170
("generic: force per-device RootFS") building non-device targets is not
possible anymore, so we can remove these obsolete handlers.

Using `make container` or, if you don't have automake/gmake on your host
system, `./scripts/container.sh` will build an image for the current
branch your are on and drop you into a shell running inside a container
using that image.

From there all tooling required to work on Gluon is available.

Supports both podman (preferred) and docker.

Sometimes it is useful to override the default version detection, for
example when local patches are applied to a repo. Allow providing a
version number using a file called .scmversion, which is the same that
the Linux kernel and U-Boot use.

A helper script is introduced to get the version from `git describe`.

Iterates over all configured modules and checks them for updates.

With the removal of ramips-rt305x, all targets come with opkg again.

Installing the script into the image allows site checks to run for
manual package installations via opkg.

All defined features need to be known at the same time, otherwise handling
a feed-provided feature definition file would add gluon-web-advanced etc.
to the package list when the corresponding feature flags appear in
GLUON_FEATURES.

Fixes: ee5ec5af ("build: rewrite features.sh in Lua")

The rewrite of the feature handling introduced multiple major bugs. One
of them was caused by the way Lua's logical operators work:

An expression of the form

    _'autoupdater' and _'web-advanced'

would return 'web-advanced' rather than the boolean true when _ returned
both strings unchanged (because the features are enabled).

As entries with more than a single feature name in their expressions did
not set no_default, Gluon would then attempt to add gluon-web-advanced to
the package selection, as web-advanced is a "pure" feature.

To fix this, and get rid of the annoying nodefault, separate handling of
"pure" feature and handling of logical expressions into two separate
functions, called feature() and when(). To simplify the feature
definitions, the package list is now passed directly to these functions
rather than in a table with a single field 'packages'.

Fixes: ee5ec5af ("build: rewrite features.sh in Lua")

Print a proper error message, rather than just

    openwrt/staging_dir/hostpkg/bin/lua: scripts/feature_lib.lua:48: bad
    argument #1 to 'setfenv' (integer expected, got nil)

Forgetting to `make update` or leaving uncommitted changes in the
repositories managed by Gluon is a recurring cause of confusion, even
for experienced developers. Let's print an obvious warning message in
this case.

site_code is evaluated early during config generation, so a site.conf
without site_code would hit this assertion that just printed 'Assertion
failed'. Add a proper error message to tell users what went wrong.

The inner assert() is removed, as it should never be hit (as site.conf
syntax will have already been validated when this script runs), and it
doesn't add anything (even without the assert, the attempt to index a
nil value would throw an error).

The current manifest format was introduced with the new autoupdater in
Gluon v2018.1. Reduce the manifest size by 70% by removing the additional
manifest lines added for backwards compatiblity.

The `features` file is converted to a Lua-based DSL.

A helper function `_` is used in the DSL; this will return the original
string for enabled features, and nil for disabled features. This allows
to use boolean operations on features without making the code too
verbose.

Besides having more readable and robust code, this also fixes the bug
that all files `packages/*/features` were evaluated instead of only
using the feature definitions of currently active feeds.

Normally, we build all nonshared packages (which includes all kernel
modules) to generate an opkg feed for later package installations by
users. On targets without opkg, this just wastes time - disable it.

(cherry picked from commit b3edfd29)

Normally, we build all nonshared packages (which includes all kernel
modules) to generate an opkg feed for later package installations by
users. On targets without opkg, this just wastes time - disable it.

Always allow options set to builtin (=y) when modular setting (=m) is
expected. This can happen when a package is added explicitly (in a
target defintion or site.mk) that is also pulled in as a dependency of
another builtin package.

Fixes: 9e23534e ("build: rework config generation")
Fixes: #2046
(cherry picked from commit 8b64517f)

Always allow options set to builtin (=y) when modular setting (=m) is
expected. This can happen when a package is added explicitly (in a
target defintion or site.mk) that is also pulled in as a dependency of
another builtin package.

Fixes: 9e23534e ("build: rework config generation")
Fixes: #2046

This change stores a Kernel with Debug-Symbols for the current
architecture in a new output directory '<outputdir>/debug'.
This allows a developer or operator of a network to store the kernel
along with the actual images. In case of a kernel oops the debug
information can be used with the script
'scripts/decode_stacktrace.sh' in the kernel source tree to get the
names to the symbols of the stack trace.

OpenWRT already provides the CONFIG_COLLECT_KERNEL_DEBUG -option that
creates a kernel with debug-symbols in the OpenWRT output directory.
This change enables this option and copies the generated kernel to the
gluon output directory.

Signed-off-by: Chrissi^ <chris@tinyhost.de>

By passing a table instead of a single string, multiple different
extensions can be specified, each refering to a separate image file
generated by OpenWrt.

This is not supported for sysupgrade (as there can only be a single
image in the format expected by OpenWrt).

manifest_aliases only make sense for sysupgrade images.

target_config.lua and target_config_check.lua don't pass a table of
callbacks anymore, so target_config_lib.lua can by simplified by moving
all the code that was in the returned function to the toplevel.

So far, we were using a sort operation on the generated .config to
implement precedence of =y packages over =m, and =m over unset.
Unfortunately, this sort not only used for packages, but for all config
lines. This made it impossible to override settings from targets/generic
in a target config when the new setting was sorted before the generic
setting.

To fix this, track configurations by their keys, so we can properly
override config keys that were set before. Value-based precedence is
only preserved for package configuration.

The config() and try_config() calls always take key and value as
separate arguments now. Strings are quoted automatically; the values
true, nil and false map to y, m and unset for tristate options. config()
can take an optional third argument to override the error message to
display when the setting fails to apply.

All existing target configs generate the same .config with the old and the
new code. The new code is also a bit faster on targets with many devices.

The OpenWrt feeds.conf.defaults contains some feeds that are commented out
and not active. Such feeds will be returned by the default_feeds.sh script
anyway and causing pseudo feeds. Limit the script to only return active
feeds, by filtering out lines starting with '#'.
This usually only applies to the OpenWrt master branch.

Signed-off-by: Sven Roederer <freifunk@it-solutions.geroedel.de>

change as per NeoRider

Certain error message (for example invalid package names) were emitted
once for each device.

The precedence of different package lists was broken since #1876,
disallowing removal of GLUON_FEATURES packages via GLUON_SITE_PACKAGES.

Including all package selections, both implicit defaults and explicit
handling in Gluon, the order of precedence is now the following:

1.  OpenWrt defaults (including target-specific defaults)
2.  Device-specific packages from OpenWrt
3.  Generic default packages (from target/generic)
4.  Target default packages (target/$(GLUON_TARGET))
5.  Removal of opkg for tiny targets
6.  Packages derived from GLUON_FEATURES + GLUON_FEATURES_$(class)
7.  GLUON_SITE_PACKAGES
8.  GLUON_SITE_PACKAGES_$(class)
9.  Device-specific packages from target/$(GLUON_TARGET)
10. Device-specific packages from GLUON_$(device)_SITE_PACKAGES

This also contains various pieces of cleanup:

- No hardcoded order of device classes for target_config.lua arguments
  anymore (in fact, the Makefile doesn't know anything about device
  classes now)
- target_conifg_lib.lua only hardcodes the fallback class for x86, no
  other occurences of specific class names
- Feature -> package list mapping is moved from Makefile to the Lua code
  as well (still implemented in Shell though)

Allows to append additional commands, for example using `||`.

We don't move the images directly, as multiple images of the same device
may have the same source image (on x86), but only delete them after a
whole device has been handled (multiple devices using the same images
must be handled using aliases or manifest aliases instead).

When adding device classes, targets without devices such as x86 were not
handled. As site and feature packages are included on such a per-device
decision, x86 images ended up without most packages.

Include a class setting for a target and include the class-packages
target-wide when this setting is configured.

Fixes 9c523650 ("build: introduce device classes")