busybox replacement idea

[Jeremy C. Reed]

This is related to the "individual software releases for third parties"
thread.

As a brief description, BusyBox provides several stripped-down Unix and
Linux tools in a single executable. I have used a few BusyBox systems
and common (coreutils or other standard "Linux") command line options
may not work and the behaviour is often different or unuseful. It has
reduced functionality. But I don't think the goal is to be complete or
user friendly but to save space. They do have a goal to be standards
compliant though. The tarball includes configuration and build rules and
then source from various projects. Depending on the configuration, there
may be up to 310 commands all contained in one binary.

NetBSD already has a similar feature: /rescue/foo. This crunchgen
solution has around 150 commands.

$ /rescue/ls -l /rescue/init
-r-xr-xr-x 152 root wheel 5113176 Feb 8 2009 /rescue/init
$ size /rescue/ls
text data bss dec hex filename
4949565 159872 2697592 7807029 772035 /rescue/ls
$ file /rescue/ls
/rescue/ls: ELF 64-bit LSB executable, x86-64, version 1 (SYSV),
statically linked, for NetBSD 5.0, stripped

(Ignore the name; all the commands are linked as a single binary.)

We should be able to just copy the existing simple and short src/rescue/
infrastructure to create the more complete replacement. Then we should
allow building this for a non-NetBSD target system. Any suggestions or
examples on how to build rescue for a non-NetBSD target system?

We may not be as small, but three benefits may be 1) user friendly
commands; 2) nice license for proprietary use (There have been
license violations reported for busybox); and 3) nice promotion of
NetBSD.

Any thoughts on this?

[t***@polynum.com]

Post by Jeremy C. Reed
[...]
We should be able to just copy the existing simple and short src/rescue/
infrastructure to create the more complete replacement. Then we should
allow building this for a non-NetBSD target system. Any suggestions or
examples on how to build rescue for a non-NetBSD target system?
We may not be as small, but three benefits may be 1) user friendly
commands; 2) nice license for proprietary use (There have been
license violations reported for busybox); and 3) nice promotion of
NetBSD.
Any thoughts on this?

This is related, but more of an use case.

I'm developing kerTeX (http://www.kergis.com/en/kertex.html), a TeX and
al. distribution for... everybody, since it is pure C89, BSD like
licence, and a _kernel_ that is a TeX system with all the core (D.E.K.'s
softwares + some essential utilities or fonts), and the ability to add
kerTeX packages at will, kerTeX dealing with its own packages (thus
relieving every host system from dealing with anything TeX related
except the hosted kerTeX by itself).

My compilation/distribution framework (R.I.S.K.) deals without problem
with systems having POSIX.2 utilities (almost all the Unices flavours,
Plan9 with APE, Mac OSX). If, for the various Windows versions,
cross-compilation could be achieved, and running the programs will work
(one exception: MetaPost uses some simple sh(1) scripts, that need
to be "batch"ed) could be done, the packaging framework uses a
subset of POSIX.2 utilities (sh(1), ed(1), sed(1), tar(1)...).

The simplest would be that these utilities be compiled for a host system
that does not provide them. On Windows, such an environment is
given by S.F.U. (but not for every system), by Cygwin (but overkill),
and Mingw (but lack at least ed(1) that I do not want to drop).

I had already contemplated taking NetBSD sources and using crunchgen(8)
for this, thus leading to a POSIX.2 limited environment on whatever
system. (Note: I use NetBSD and Plan9 and not Windows... So this is for
"others".)

Just to say that this could be useful also to provide this for non POSIX
systems, but letting compilers, assemblers and linkers be the host ones
(speaking about Windows, Microsoft does publish a compiler framework
that a Windows user can download; so the Unix/POSIX way of dealing with
sources could be used with the utilities and the native compiler
framework).

So this could be of a wider use than "only" Busybox like replacement.

FWIW,

[Aleksey Cheusov]

Post by Jeremy C. Reed
We should be able to just copy the existing simple and short src/rescue/
infrastructure to create the more complete replacement. Then we should
allow building this for a non-NetBSD target system. Any suggestions or
examples on how to build rescue for a non-NetBSD target system?

You will not be able to build NetBSD utils on non-NetBSD platforms due
to some BSD-isms like fgetln, strl* and others. Some time ago I planed
to start nbutils project (like core-utils but with NetBSD uselevel) for
the same purposes -- NetBSD's tools portable to at least *BSD, Linux,
Solaris and Darwin. For this purpose I wanted to use my own build
framework devel/mk-configure. As mk-configure is actually a fork of
NetBSD mk-files and its idealogy is the same I think it should be
friendly for all BSD developers and to some extent compatible with
NetBSD mk files. So, making a source code portable is not hard at
all.

The only problem I see is a periodic syncing with NetBSD CVS. Not that
it is not possible but it would be much better if we could "fork" (in
terms of Distributed SCM) src/ subdirectories into a single project. In
this case "merging" would be easier. Though, I'm not sure any DSCM
system provides such functionality.

In this presentation (page 39)
http://mova.org/~cheusov/pub/mk-c/mk-c.pdf
you can see how I made portable version of AWK from NetBSD CVS.

Source code for nbawk is available here
http://mova.org/~cheusov/pub/mk-c/nbawk/

P.S.
https://github.com/chneukirchen/obase

[Julio Merino]

Post by Aleksey Cheusov
You will not be able to build NetBSD utils on non-NetBSD platforms due
to some BSD-isms like fgetln, strl* and others.

You can either patch the code to not use NetBSD-isms, depend on
libnbcompat (as done in pkgsrc) or add conditional-specific code
variants for other systems.

Post by Aleksey Cheusov
The only problem I see is a periodic syncing with NetBSD CVS. Not that
it is not possible but it would be much better if we could "fork" (in
terms of Distributed SCM) src/ subdirectories into a single project. In
this case "merging" would be easier. Though, I'm not sure any DSCM
system provides such functionality.

One of the ideas that surfaced a while ago was to put such code into
src/internal/<project>/ or similar, and maintain it there in a form
that can readily be shipped externally.

[Aleksey Cheusov]

Post by Julio Merino

Post by Aleksey Cheusov
You will not be able to build NetBSD utils on non-NetBSD platforms due
to some BSD-isms like fgetln, strl* and others.

You can either patch the code to not use NetBSD-isms,

Very bad way I think.

Post by Julio Merino
depend on libnbcompat (as done in pkgsrc) or add conditional-specific
code variants for other systems.

I meant that for things like nbcompat we'll need something more than
plain mk files and bmake. I proposed mk-configure for autoconf-like
functionality as it is very close to original NetBSD build system.

[Alan Barrett]

Depending on the configuration, there may be up to 310 commands
all contained in one binary [with busybox].
NetBSD already has a similar feature: /rescue/foo. This
crunchgen solution has around 150 commands.

I believe that crunchgen has no fixed limits to the number of
commands.

We should be able to just copy the existing simple and short
src/rescue/ infrastructure to create the more complete
replacement.

Yes, of course. However, this often requires adding a few
conditional lines to the Makefile for the program to be built, and
if you want to omit some features from the crunched version of the
program then you have to add conditional code to the source of the
program itself.

For example, the build infrastructure for /rescue instructs
crunchgen to set SMALLPROG in the environment; sbin/init/Makefile
uses that to change the CPPFLAGS and libraries; and
sbin/init/init.c uses the preprocessor definitions to
conditionally exclude some features.

Then we should allow building this for a non-NetBSD target
system. Any suggestions or examples on how to build rescue for a
non-NetBSD target system?

This is the only non-trivial part.

We may not be as small, but three benefits may be 1) user
friendly commands; 2) nice license for proprietary use (There
have been license violations reported for busybox); and 3) nice
promotion of NetBSD.
Any thoughts on this?

It's not difficult to have more than two levels of "the
full-featured command" and "a smaller version omitting some
features". Different invocations of crunchgen can set different
environment variables (akin to the existing SMALLPROG variable)
which can be fed through the Makefils and CPPFLAGS to achieve
finer control over what features to include or exclude. (For
example, I use crunchgen to build images in which init is
full-featured, whereas the standard NetBSD /rescue contains an
init with reduced functionality.)

--apb (Alan Barrett)

[Thor Lancelot Simon]

Post by Jeremy C. Reed
We should be able to just copy the existing simple and short src/rescue/
infrastructure to create the more complete replacement. Then we should
allow building this for a non-NetBSD target system. Any suggestions or
examples on how to build rescue for a non-NetBSD target system?

You're kidding about "simple", right?

Crunchgen is probably the most fragile component of the entire build
system.

Thor

[Thorsten Glaser]

Post by Jeremy C. Reed
As a brief description, BusyBox provides several stripped-down Unix and

There’s beastiebox by imil, your fellow NetBSD® developer.

bye,
//mirabilos