Upstream introduced two different methods for determining the state of a
package within the OpenWrt buildsystem. While both are based around the
md5 hash-function, one taks filename & mtime into account while the
other one uses the actual md5 hash of the file-content.

Ever wondered why Gluon suddenly took considerably longer to build?
The messy part is how the build-system chooses which method to use. This
is based around the AUTOREMOVE configuration. Gluon sets this variable
conditionally when built with GLUON_AUTOREMOVE set to 1.

Enter the Gluon build-system. It first compiles Lua, without the
AUTOREMOVE configuration passed to OpenWrt. This compiles the packages
with the old hash-method based around filename & mtime. Afterwards, it
builds with AUTOREMOVE enabled, changing the hash-function and
rebuilding all host-packages.

Fix this by setting AUTOREMOVE for both build-processes according to the
setting of GLUON_AUTOREMOVE.

Link: https://github.com/openwrt/openwrt/commit/53a08e37437972ba0a8fbf953a93a70a6b784ef4



Signed-off-by: David Bauer <mail@david-bauer.net>

We don't want OpenWrt packages to replace Gluon packages by accident;
the same logic applies to packages from site feeds.

This gives site feeds a higher precedence than gluon/packages, but that
is unlikely to cause any issues. Gluon base packages still have an even
higher precedence.

The command was more complicated than necessary, and somehow contained
two redirects of stdout.

Fixes access to the build directory mounted into the container when UIDs
don't match.

For Docker, HOME is set in addition to get a more useful value than /
and for consistency with Podman.

Adds `lint-editorconfig` make target.

Co-authored-by: Jan-Niklas Burfeind <git@aiyionpri.me>
version bump to 2.6.0

Device-specific package additions could generate `CONFIG_PACKAGE_...=m`
lines, which would override `CONFIG_PACKAGE_...=y` lines inserted by
OpenWrt for default packages (as Gluon did not know about these default
packages). This resulted in the unintended removal of such packages from
other devices that did not contain the same package in their device
package lists.

Avoid this issue by explicitly adding OpenWrt's target default package
list to the front of Gluon's target package list.

OpenWrt's default package list contains the package "nftables", which is
a virtual package provided by "nftables-json" and "nftables-nojson".
Explicitly handle this case, otherwise our config check will fail when
we extend our default package list with the one from OpenWrt.

Another leftover from legacy OpenWrt targets, which we don't support
anymore.

No legacy OpenWrt targets exist anymore which require the .config profile
name to differ from the image name.

The following features are available in Busybox ash, so we don't need to
warn about them for runtime scripts:

- local keyword
- echo -n / -e
- String indexing

These warnings are new in shellcheck 0.7.2, which would otherwise fail
for various scripts.

Resolves #2362

All our targets use the OpenWrt device abstraction. Since commit
6ba58c9b17c90e41b521d796ab76e5723ee017170
("generic: force per-device RootFS") building non-device targets is not
possible anymore, so we can remove these obsolete handlers.

Using `make container` or, if you don't have automake/gmake on your host
system, `./scripts/container.sh` will build an image for the current
branch your are on and drop you into a shell running inside a container
using that image.

From there all tooling required to work on Gluon is available.

Supports both podman (preferred) and docker.

Sometimes it is useful to override the default version detection, for
example when local patches are applied to a repo. Allow providing a
version number using a file called .scmversion, which is the same that
the Linux kernel and U-Boot use.

A helper script is introduced to get the version from `git describe`.

Iterates over all configured modules and checks them for updates.

With the removal of ramips-rt305x, all targets come with opkg again.

Installing the script into the image allows site checks to run for
manual package installations via opkg.

All defined features need to be known at the same time, otherwise handling
a feed-provided feature definition file would add gluon-web-advanced etc.
to the package list when the corresponding feature flags appear in
GLUON_FEATURES.

Fixes: ee5ec5af ("build: rewrite features.sh in Lua")

The rewrite of the feature handling introduced multiple major bugs. One
of them was caused by the way Lua's logical operators work:

An expression of the form

    _'autoupdater' and _'web-advanced'

would return 'web-advanced' rather than the boolean true when _ returned
both strings unchanged (because the features are enabled).

As entries with more than a single feature name in their expressions did
not set no_default, Gluon would then attempt to add gluon-web-advanced to
the package selection, as web-advanced is a "pure" feature.

To fix this, and get rid of the annoying nodefault, separate handling of
"pure" feature and handling of logical expressions into two separate
functions, called feature() and when(). To simplify the feature
definitions, the package list is now passed directly to these functions
rather than in a table with a single field 'packages'.

Fixes: ee5ec5af ("build: rewrite features.sh in Lua")

Print a proper error message, rather than just

    openwrt/staging_dir/hostpkg/bin/lua: scripts/feature_lib.lua:48: bad
    argument #1 to 'setfenv' (integer expected, got nil)

Forgetting to `make update` or leaving uncommitted changes in the
repositories managed by Gluon is a recurring cause of confusion, even
for experienced developers. Let's print an obvious warning message in
this case.

site_code is evaluated early during config generation, so a site.conf
without site_code would hit this assertion that just printed 'Assertion
failed'. Add a proper error message to tell users what went wrong.

The inner assert() is removed, as it should never be hit (as site.conf
syntax will have already been validated when this script runs), and it
doesn't add anything (even without the assert, the attempt to index a
nil value would throw an error).

The current manifest format was introduced with the new autoupdater in
Gluon v2018.1. Reduce the manifest size by 70% by removing the additional
manifest lines added for backwards compatiblity.

The `features` file is converted to a Lua-based DSL.

A helper function `_` is used in the DSL; this will return the original
string for enabled features, and nil for disabled features. This allows
to use boolean operations on features without making the code too
verbose.

Besides having more readable and robust code, this also fixes the bug
that all files `packages/*/features` were evaluated instead of only
using the feature definitions of currently active feeds.

Normally, we build all nonshared packages (which includes all kernel
modules) to generate an opkg feed for later package installations by
users. On targets without opkg, this just wastes time - disable it.

(cherry picked from commit b3edfd29)

Normally, we build all nonshared packages (which includes all kernel
modules) to generate an opkg feed for later package installations by
users. On targets without opkg, this just wastes time - disable it.

Always allow options set to builtin (=y) when modular setting (=m) is
expected. This can happen when a package is added explicitly (in a
target defintion or site.mk) that is also pulled in as a dependency of
another builtin package.

Fixes: 9e23534e ("build: rework config generation")
Fixes: #2046
(cherry picked from commit 8b64517f)

Always allow options set to builtin (=y) when modular setting (=m) is
expected. This can happen when a package is added explicitly (in a
target defintion or site.mk) that is also pulled in as a dependency of
another builtin package.

Fixes: 9e23534e ("build: rework config generation")
Fixes: #2046

This change stores a Kernel with Debug-Symbols for the current
architecture in a new output directory '<outputdir>/debug'.
This allows a developer or operator of a network to store the kernel
along with the actual images. In case of a kernel oops the debug
information can be used with the script
'scripts/decode_stacktrace.sh' in the kernel source tree to get the
names to the symbols of the stack trace.

OpenWRT already provides the CONFIG_COLLECT_KERNEL_DEBUG -option that
creates a kernel with debug-symbols in the OpenWRT output directory.
This change enables this option and copies the generated kernel to the
gluon output directory.

Signed-off-by: Chrissi^ <chris@tinyhost.de>

By passing a table instead of a single string, multiple different
extensions can be specified, each refering to a separate image file
generated by OpenWrt.

This is not supported for sysupgrade (as there can only be a single
image in the format expected by OpenWrt).

manifest_aliases only make sense for sysupgrade images.

target_config.lua and target_config_check.lua don't pass a table of
callbacks anymore, so target_config_lib.lua can by simplified by moving
all the code that was in the returned function to the toplevel.

So far, we were using a sort operation on the generated .config to
implement precedence of =y packages over =m, and =m over unset.
Unfortunately, this sort not only used for packages, but for all config
lines. This made it impossible to override settings from targets/generic
in a target config when the new setting was sorted before the generic
setting.

To fix this, track configurations by their keys, so we can properly
override config keys that were set before. Value-based precedence is
only preserved for package configuration.

The config() and try_config() calls always take key and value as
separate arguments now. Strings are quoted automatically; the values
true, nil and false map to y, m and unset for tristate options. config()
can take an optional third argument to override the error message to
display when the setting fails to apply.

All existing target configs generate the same .config with the old and the
new code. The new code is also a bit faster on targets with many devices.