handle errors on sha512sum - also handle awk errors inside
the mini subshell, and provide overall error handling.
we know that the project.hash file should always exist, and
always be read no matter what; technically, the find command
that proceeds it might not yield any results, but an empty
file would then be produced.
the edge case of an empty file would have lead to an error
beforehand, when configuring the project in function,
configure_project(), so we've already got that covered.
Signed-off-by: Leah Rowe <leah@libreboot.org>
when reading old_pjhash, we need to error out where a read
error occurs. such an error is unlikely, but could occur under
certain edge cases.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Don't do no-op if it fails; fall back to "clean" instead,
and fail if that fails.
The no-op was there was not all projects have distclean,
but we do intend for them all to be cleaned.
We mitigate further error by only running make-clean if
a makefile exists.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Don't copy the files directly, because we might be doing
this from a work directory that has no files; in this case,
generic "unknown" variables are used, without generating
any files, so the current logic would produce an error.
However, we do need to create those dot files, because
we then rely on them for building release binaries.
The new logic maintains current behaviour, while fixing
this technical edge-case scenario via mitigation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
stick it in git_prep, which both single- and multi-tree
projects will use, when downloading git repositories.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The following execution will result in another printf
that says exactly what is being downloaded.
There is no need to inform the user twice about
what is being downloaded.
Signed-off-by: Leah Rowe <leah@libreboot.org>
A git-pull is performed immediately after git-fetch.
Git-pull already performs git-fetch as a prerequisite.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the checks at the end of the function are mostly
superfluous, because bad_checksum() is immediately
called just beforehand, and performs the same checks.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the current behaviour is a relic from the older lbmk
design, before recent auditing.
the current logic would cause xbmk to continue execution,
going into a child process with .git/ being a symlink.
The .git/ directory should never be a symlink, because
it is extremely error-prone.
Signed-off-by: Leah Rowe <leah@libreboot.org>
We rely on a non-zero exit on other try_ commands, which
works fine there because we then check the file afterward
and error out accordingly.
For git repositories, we assume that both mirrors are
identical and therefore once we get to the first clone
attempt, we assume that it must succeed.
Therefore, if it does not succeed, we must fail. This fixes
a regression I found in testing, where sometimes a failed
patching attempt would not result in an error exit, and
would therefore result in broken sources being present.
In practise, I always very closely watch the terminal when
testing xbmk, especially when updating project patches, so
we probably didn't introduce any broken sources in practice.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This code was introduced to provide fault tolerance,
so that if I forgot to manually update the configs
myself, builds would still succeed, e.g. coreboot
builds.
However, there have been cases in the past where this
introduces settings we don't want, and in general we
do want to know when there is an error in the configs.
The policy should always be: fail early, fail hard.
This also mitigates bugs in U-Boot's build system; for
example, when I last attempted to update the U-Boot
tree for x86, make-oldconfig introduced a lot of junk
settings unrelated, which then introduced code that
would brick the board if you tried it on one, e.g.
it broke booting most Linux kernels via bootflow.
With this change, U-Boot will be easier to handle,
which normally requires manual configuration; the
automated make-oldconfig reconfiguration feature
breaks U-Boot. This will no longer occur, since we
no longer run it manually.
On the other hand, this feature has also prevented
other disastrous bugs in the past, such as when I
forgot to properly set the SPD size on T480; it was
set to 256 bytes, not 512 as is correct. Therefore,
this new design change means I must also be more
vigilant about config changes in project trees.
Signed-off-by: Leah Rowe <leah@libreboot.org>
it mainly does general tasks, like handling utils
and enabling ccache. the vfiles are a small part.
rename the function accordingly. it is called by
premake, so let's call it corebootpremake.
this change will also make sense when cherry-picked
into cbmk, which does not handle vfiles at all.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we simply do not need to run the make-oldconfig command
at all, and after removing it, the "cook" function seemed
quite redundant so i merged it with mkvendorfiles()
Signed-off-by: Leah Rowe <leah@libreboot.org>
define it with a single variable, rather than several.
this allows several checks to be greatly simplified.
Signed-off-by: Leah Rowe <leah@libreboot.org>
In practise, coreboot can set this bit at build time.
We also use ME Soft Temporary Disable by default, on
this platform.
We also use me_cleaner by default, so the me.bin file
added to flash only contains the code that would run
with HAP set anyway.
Therefore, this change is of little practical consequence,
but as a friend put it to me, this change is most technically
correct.
And I'm all about technical correctness.
Signed-off-by: Leah Rowe <leah@libreboot.org>
We only need the Kabylake version. We can safely
remove the other ones, thereby significantly
reducing the size of the lbmk release archive.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Right now, if cache/clone/PROJECT/ already exists,
the logic for pulling new changes doesn't execute,
and neither does the logic for updating remotes.
This is bad when updating revisions, because then
manual updating is required, defeating the purpose
of xbmk's own automation in this regard.
Fix it by only checking the cached download on files,
not Git repositories; the try_git function itself will
already perform this check, before updating remotes
and pulling in new commits from upstream.
The updating only happens when a given target directory
doesn't exist, e.g. src/flashprog/ or src/grub/default/,
so this won't slow down release builds for example.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Otherwise, an "unknown" version number is created.
This regression was caused by the recent optimisation
that reduces the amount of extra work done by init.sh
on child instances of xbmk.
As a result of those changes, now release.sh has to
do some minor initialisation of its own, such as this.
Signed-off-by: Leah Rowe <leah@libreboot.org>
also pcsx-redux
this way, commands like "./mk -u" without argument
will not fail. these fake makefile commands do nothing.
otherwise, an error errors because their makefiles
do not define these options.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Otherwise, ./mk -d (without arguments) fails for GRUB,
which first requires running autoconf to get a Makefile.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This brings in several changes from upstream:
* 73d1c959e cryptocheck: Add --quiet option
* dbc0eb5bd disk/cryptodisk: Wipe the passphrase from memory
* 301b4ef25 disk/cryptodisk: Add the "erase secrets" function
* 23ec4535f docs: Document available crypto disks checks
* 10d778c4b commands/search: Add the diskfilter support
* 7a584fbde disk/diskfilter: Introduce the "cryptocheck" command
* ed691c0e0 commands/search: Introduce the --cryptodisk-only argument
* c448f511e kern/rescue_reader: Block the rescue mode until the CLI authentication
* 4abac0ad5 fs/xfs: Fix large extent counters incompat feature support
This commit is of particular interest:
* dbc0eb5bd disk/cryptodisk: Wipe the passphrase from memory
Signed-off-by: Leah Rowe <leah@libreboot.org>
This reverts commit fb7aaa78bb.
it caused a few issues. will re-do later
the old code isn't really broken, just inefficient, because
several files are scanned twice, but in practise the overhead
isn't that great
The error occurs sometimes, when bruteforcing me.bin:
ERROR ./mk: Unhandled error for: mv /home/user/lbmk/tmp/me.bin /home/user/lbmk/cache/tmpdl/check
This revert should fix the issue, for now.
i'm adding characters to 7ztest, which isn't being passed
on through because everything runs in subshells; the next
pass would default back to the original string, so a given
file may be checked multiple times.
fix this by mitigation; use the random string from mktemp
as a suffix instead.
in practice, this has not affected performance much, but it
will nevertheless avoid unnecessary work by xbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't skip if the URL is empty. throw an error instead.
i decree that all links must be properly initialised, because
that is the design of lbmk. where only one link is provided,
such as in a local copy operation, the second would succeed no
better than the first so two identical paths are given.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the intent once again is that this for loop shall
return, with zero status, if success is observed.
otherwise, the loop breaks and an error is thrown.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The idea in this function is that if a file or repo is
successfully handled, a return will be performed from the
loop.
If the loop exits for any reason, an error is thrown. The
current code is probably fine, but I can forsee future
modifications possibly causing bugs here.
Make it unambiguous, by always throwing an error if execution
reaches the end of the function.
Signed-off-by: Leah Rowe <leah@libreboot.org>
Because of how sh works, having just the [] line causes
sh to exit, annoyingly without an error message, but it
does cause a non-zero exit.
This bug will have already been triggering, before I added
the recent error handling on files for this for loop.
also do it to the other loop in lib.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
I never really intended for this to be configurable,
but the cache directory is also used during release
builds.
There's too much that can go wrong, letting the user
decide where their cache is. Simplify it by hardcoding.
Signed-off-by: Leah Rowe <leah@libreboot.org>
already present on a few other config files, e.g. arch
i noticed on debian-experimental that i needed to explicitly
install it, whereas it was implicitly installed on debian 12
Signed-off-by: Leah Rowe <leah@libreboot.org>
it's just two lines, and we want much more granular
control of where the lock is enforced. it should be
JUST after confirming that the instance is a parent.
it is at this moment that we should bail if a lock
file exists, because this signals that another instance
of xbmk is running.
Signed-off-by: Leah Rowe <leah@libreboot.org>
once again, we are being stricter in child instances.
we must ensure that these variables are set by xbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we no longer rely on the .git version being
read by child instances, so we MUST ensure
that it is being read.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't update them on child instances, since it's a waste
of time; the lock file prevents further execution, so we
are just wasting time writing to disk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we don't need to read or write a file at all, in that case.
we only then need to generate one if running ./mk release.
the scenario in which no .git and no version files exist
is when someone grabs the build system from a snapshot
generated by e.g. forgejo instances. it's ill advised, so
we advise against it, but it is mitigated in code.
Signed-off-by: Leah Rowe <leah@libreboot.org>
That way, unnecessary work is avoided on child instances.
Of course, the current check assumes that TMPDIR wasn't
already set by a wily user before running lbmk, but then
those sorts of users probably know what they're doing.
If they don't know, they will soon find out. Therefore, I
have added additional checks on child instances, preventing
the build system from running if XBMK_CACHE is not set; if
it isn't, then that could very easy lead to certain system
files being overwritten.
The user must never know what happens if XBMK_CACHE is unset.
We simply will not allow it.
Signed-off-by: Leah Rowe <leah@libreboot.org>
all this function does now is create the python symlink,
based on work that was already performed in set_pyver
Signed-off-by: Leah Rowe <leah@libreboot.org>
Do it after the creation of xbmkpath.
This avoids performing an unnecessary check, since
PATH will have already been corrected for child
instances; Python will already be correct there.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we mkdir -p xbmklocal, only to remkdir it immediately
afterward, which is the intended behaviour; on parent
instances, xbmklocal is to be re-created fresh.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this function now simply creates directories that lbmk
will use, rather than creating specific directories.
Signed-off-by: Leah Rowe <leah@libreboot.org>
we must only set this in the parent instance, not
child instances. this prevents the variable from
being over-populated with repeated entries.
Signed-off-by: Leah Rowe <leah@libreboot.org>
this way, initialisation will not be performed erroneously
while another parent instance of lbmk is running.
Signed-off-by: Leah Rowe <leah@libreboot.org>
This is earlier than the current check, thus preventing
the initialisation of a git repository and/or the recreation
of xbmktmp and xbmklocal by erroneous parent executions of lbmk
while another parent is running - the latter of which could have
caused a massively unpredictable build failure, so this is also
a pre-emptive bug fix, fixing all kinds of weird bugs.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i wasn't ok having that variable initialisation and
then the commands on the same line. it looks messy.
having the commands on a separate line makes the code nice
to read, so let's separate them.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the user doesn't care where the temporary git repo is
git shows that information anyway, in the git clone command
Signed-off-by: Leah Rowe <leah@libreboot.org>
we really only need it there, because the context is
for release archives. normal use of the git repository
doesn't matter in the context of deletions, because that
will not be distributed. only the result of ./mk release
will be distributed.
the builds produced will not change as a result of this,
for people using the normal git repository, because the
files in question are never used anyway, in our configs.
this is being done to make working on local repos easier.
Signed-off-by: Leah Rowe <leah@libreboot.org>
otherwise, ./mk -b (without argument) will fail, on release
archives. also, perhaps i should add an mkhelper to build it?
Signed-off-by: Leah Rowe <leah@libreboot.org>
this is the mkdir call that createsn the directory where
a cached git repository is moved to, during creation.
Signed-off-by: Leah Rowe <leah@libreboot.org>
On release archives, I overlooked the previous change to
downloads, during the recent implementation of extra safety
checks. I previously checked there whether the variable named
CONFIG_KBC1126_FIRMWARE was defined, and grabbed both; now I
check CONFIG_KBC1126_FW1 and CONFIG_KBC1126_FW2 separately,
grabbing each file separately.
This patch replicates that change for insertions. Otherwise,
hash verification on ROM images will fail, when running the
inject script on release images.
Downloading was being done, reliably, and the extracted files
were correct, so there was no danger if the user was building
from source and flashing that way.
However, checksum verification on full images failed when
inserting into archives. This is not because the files were
wrong; they were *correct*. However, the EC firmware was not
being inserted *at all* on HP EliteBooks, because of this
oversight. The check is now based on whether the paths to
the files themselves are defined, not whether EC firmware
is enabled in the coreboot config; the latter is implied.
With this patch, vendor file insertion once again works
perfectly, without error, on every board. There was no real
danger for users, just a minor inconvenience. Sorry!
Signed-off-by: Leah Rowe <leah@libreboot.org>
the exit from mkdst can also be non-zero if mv or cp
failed, but there's no way to handle that reliably.
therefore, the checksum verification should be done
one final time, to compensate.
Signed-off-by: Leah Rowe <leah@libreboot.org>
I currently check the downloaded files e.g. .exe file, but
then I don't check - or even define - sha512sums for the
files extracted from them e.g. me.bin
This patch fixes that. It also caches the hashed files, so
that extraction is faster on a re-run - this makes release
builds go faster, when running ./mk release
If a checksum is not defined, i.e. blank, then a warning is
given, telling you to check a specific directory. This way,
when adding new vendor files, you can add it first without
specifying the checksum, e.g. me.bin checksum. Then you can
manually inspect the files that were extracted, and define it,
then test again.
In a given pkg.cfg for config/vendor, the following variables
are now available for use:
FSPM_bin_hash for fsp m module
FSPS_bin_hash for fsp s module
EC_FW1_hash for KBC1126 EC firmware (1st file)
EC_FW2_hash for KBC1126 EC firmware (2nd file)
ME_bin_hash for me.bin
MRC_bin_hash for mrc.bin (broadwell boards)
REF_bin_hash for refcode (broadwell boards)
SCH5545EC_bin_hash for sch5545 firmware (Dell Precision T1650)
TBFW_bin_hash for Lenovo ThunderBolt firmware (e.g. T480/T480s)
E6400_VGA_bin_hash for Dell E6400 Nvidia VGA ROM
In practise, most people use release archives, and the
inject script, so I knew those were reliable, because the ROM
images were hashed prior to removing files. This patch benefits
people using lbmk.git directly, without using release files,
because now they know they have a valid file e.g. me.bin
Previously, only the download was checked, not the extracted
files, which meant that the only thing preventing a brick was
the code not being buggy. Any number of bugs could pop up in
the future, so this new level of integrity will protect against
such a scenario, and provide early warning prompting bug fixes.
Signed-off-by: Leah Rowe <leah@libreboot.org>
in a few places, we use the presence of a file found
by fx_ to cause an exit, but the command that runs
looks something like:
exit 1 "string"
this yields an error, and a non-zero exit, because of
too many arguments to "exit", but we wanted a non-zero
exit anyway.
nevertheless, this is incorrect.
to fix it, eval is used instead. if the never-going-to-exist
condition one day exists where exit 1 actually returns, not,
you know, exits, we will use err instead, with the string
as argument.
this should be fine. it's a bit hacky, but so is fx_, and
it works. fx_ is used in several places to keep the sloccount
down, providing a common way to perform while loops on the
output of a command; that is its only purpose..
Signed-off-by: Leah Rowe <leah@libreboot.org>
more specifically, re-write it so that it can be called with fx_
this means that the single-tree check for nuke.list can be made
much simpler
Signed-off-by: Leah Rowe <leah@libreboot.org>
This way, we can use x_ which will then print the command
that failed, if we need to debug future errors.
Signed-off-by: Leah Rowe <leah@libreboot.org>
I overlooked this in a previous patch. It doesn't really
matter, since we're operating on a file anyway, but it's
not correct.
Files should have rm -f on them, not rm -Rf, for deletion.
Signed-off-by: Leah Rowe <leah@libreboot.org>
We already do what the old code does in setcfg, by
virtue of the fact that the st variable is later
checked, after loading this config conditionally,
where the st variable is otherwise blank.
We can avoid the unnecessary work after loading
the config, by returning if the config is absent.
Signed-off-by: Leah Rowe <leah@libreboot.org>
We are calling xbmkget in the same way, whether it's
a subfile or subrepo.
Rename these variables to subcurl and subgit, so that we
can call xbmkget unconditionally.
Signed-off-by: Leah Rowe <leah@libreboot.org>
they were always re-downloading every time.
i've basically re-written most of xbmkget.
there was some erroneous conditions under which
it wrongly deleted the cached file, resulting in
it being downloaded again.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The result of the printf statement is sorted, making
it do binaries first, which results in a lot of junk
files then being present inside the source archive.
Signed-off-by: Leah Rowe <leah@libreboot.org>
it now handles more than just git, and i forsee
it handling even more in the future, e.g. rsync,
ftp, bittorrent.
Signed-off-by: Leah Rowe <leah@libreboot.org>
And this time it works.
I'm now calling xbmkget() which in turn calls tmpclone(),
instead of me calling tmpclone() directly.
The git-pull is done on both remotes, regardless of whether
the first succeeds. This way, if I forgot to update a mirror,
downloads would probably still work.
This also fixes an issue people were having, for example where
the gnulib repository of GRUB was always being downloaded
every time.
I'm using a new directory, XBMK_CACHE/clone, instead
of XBMK_CACHE/repo (which I used before), in case people
still have the old caches from before.
Signed-off-by: Leah Rowe <leah@libreboot.org>
don't move to the real directory until the work
is done.
that way, a re-try can be done, while analysing
the old files. it is created based on the tmpdir,
under XBMK_CACHE/
Signed-off-by: Leah Rowe <leah@libreboot.org>
We don't need to call it from git.sh, because it's
only being done when building a release anyway,
and we already run rmgit when doing a release.
The function itself is only two simple fx_ calls,
so we can just do that from build_release().
Signed-off-by: Leah Rowe <leah@libreboot.org>
in cbmk, it's only used from there.
in lbmk, it's also used from vendor.sh.
however, i plan to further expand git.sh at
some point, tidying it up so that git cloning
is also done from xbmkget, with dlop=git and
git.sh would then be renamed to get.sh
Signed-off-by: Leah Rowe <leah@libreboot.org>
group the cbfs command to the extract command, since they
are related. this makes it clearer that the following
command to extract refcode is unrelated.
Signed-off-by: Leah Rowe <leah@libreboot.org>
it was too complicated. most of the logic has been moved
to a new function, try_file()
the for loop is handled by xbmkget(), whereas each try
is now handled in try_file()
Signed-off-by: Leah Rowe <leah@libreboot.org>
split the actual bootstrapping to getvfile()
setvfile only sets the config, but then it will
call getvfile() to act on that config.
Signed-off-by: Leah Rowe <leah@libreboot.org>
i moved out more code to vendor.sh, to reduce the
amount of lbmk-only code on inject.sh
this should reduce the number of merge conflicts
even further, when cherry picking from lbmk to cbmk.
in particular, vendor file insertion is now handled
entirely through the "setvfile" function, instead
of from inject.sh, which seems counterintuitive,
but remember that inject.sh also does MAC addresses.
therefore, the inject.sh script is now primarily for
inserting MAC addresses, and handles vendor downloads
in a slightly more convoluted way, but still easy
enough to understand if you read it a bit.
Signed-off-by: Leah Rowe <leah@libreboot.org>
otherwise, we create empty directories where build.list
doesn't exist, like on coreboot.
we already create a directory when needed, when actually
copying elf files, so let's just leave it at that.
Signed-off-by: Leah Rowe <leah@libreboot.org>
to the extent feasible, keep lbmk-specific parts on
inject.sh to a minimum. this will later be used to
re-sync cbmk's inject.sh with lbmk's, because cbmk's
one doesn't handle vendor files.
the way this is designed now, with this patch, will
make cherry-picking lbmk to cbmk easier in the future,
when keeping this part of cbmk in sync with lbmk.
Signed-off-by: Leah Rowe <leah@libreboot.org>
generally go for a more linear function order, and
split up any functions.
the objective is to have functions only suitable to
libreboot be separate. more splitting will be done,
and eventually the vendor-download functions will be
split into a new file, as will several other functions.
this is being done as part of an effort to bring the
libreboot and canoeboot versions of inject.sh in sync,
so that from now on, cherry picking between the two
projects will produce fewer merge conflicts and require
a lesser amount of post-merge maintenance.
some other minor cleanup has also been done; for example,
the "need_files" variable is redundant and was removed.
Signed-off-by: Leah Rowe <leah@libreboot.org>
many places in lbmk used err, because older versions
of x_ did not handle globbing properly.
however, use of x_ is preferable on trivial commands.
the only time err() should be called is what it has
to be, when x_ can't work, or when a more useful error
message is needed, for context.
Signed-off-by: Leah Rowe <leah@libreboot.org>
that way, the Intel GbE device can be enabled there,
and only then would the refcode file be copied.
otherwise, the current behaviour would leave buggy
refcode in place, if the dd command failed.
Signed-off-by: Leah Rowe <leah@libreboot.org>
use of the e function would slow down execution,
and it's mostly unnecessary in this case.
the e function is only needed if we want to confirm
via user message that a file exists. that is not
needed here.
Signed-off-by: Leah Rowe <leah@libreboot.org>
the current test allows a further extraction after
running mecleaner, even if me.bin was found.
further, any recursive calls that exit non-ze
don't lot the loop acthually stop, unless we
subshell that too, otherwise fx_ is returned to
return 0 when a given command it runs returns 1,
or more specifically: the for loop in x_ breaks.
this is by design, and there's not much that can
be done, but this patch should pseed up extraction
a little bit, when dealing with intel me files.
Signed-off-by: Leah Rowe <leah@libreboot.org>
that way, with set -u -e, we aren't risking some
buggy sh implementations from causing an error exit
where it shouldn't.
Signed-off-by: Leah Rowe <leah@libreboot.org>
The idea with mk is that it's meant to basically be a
stub for running everything else, while mainly having
the trees logic within it (what was once script/trees).
Signed-off-by: Leah Rowe <leah@libreboot.org>
2025-05-08 23:28:49 +01:00
165 changed files with 1563 additions and 1348 deletions
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
