gotadmin cleanup: don't delete pack files that are too young similar to what we do for loose objects, avoid deleting pack files that were created "too soon" unless -a is given. This prevents races when gotadmin load, got fetch or gotd are installing a new pack file and a concurrent gotadmin clean attempts to remove it. ok stsp

purge_redundant_pack(): don't special-case .pack or .idx leftover, the error handling of unlink() a couple of lines below used to carry a similar special-casing but was dropped; drop it here too.

take reachability in consideration when cleaning redundant packfiles This wraps the cleaning up of loose objects and redundant pack files under a new functions, making the _prepare() and _complete() functions unnecessary. It walks the reachable commits unconditionally since that information is always needed and adapt how we purge loose objects after this change. The progress function is changed too and we log `ncommits' first, followed by the number of loose objects, followed by the rest. Pack files are now considered redundant when all their objects are provided by a bigger pack or are unreachable. ok stsp

unfold a line

use _POSIX_HOST_NAME_MAX from <limits.h> for portability ok stsp

rate-limit redundant pack cleanup progress output

fix cleanup progress reporting output

add a lock for the cleanup operation This adds the functions got_repo_cleanup_prepare() and _complete() to lock/unlock the repository to prevent multiple `gotadmin cleanup' or `git gc' operations to run in parallel. improvements and ok stsp@

gotadmin cleanup: remove redundant pack files improvements and ok stsp@

add got_repo_get_object_format() and use it to avoid hardcoding the digest type in a few places. ok stsp@

provide functions to parse/serialize different hashes it abstracts over the hash type and ensures that object ids are zero'ed before their sha1 digest is written. Needed by the incoming sha256 support. ok stsp@

rename lib/sha1.c to lib/hash.c It will soon grow functions to deal with sha256 too. stsp@ agrees.

include sha2.h too where sha1.h is included In preparation for wide sha256 support; stsp@ agrees. Change done mechanically with find . -iname \*.[cy] -exec sam {} + X ,x/<sha1\.h>/i/\n#include <sha2.h>

gotadmin pack: add a -D flag to force using ref-delta Intended mostly for the regress suite, so we'll be able to test also the ref-delta code paths. ok stsp@

gotd, gotadmin: install packfiles and index files as 0444 gotd used 0600 (due to mkstemps(3)), gotadmin 0644; change it to 0444 since packfiles shouldn't change once created. Mirrors what git does. ok stsp@

use mkstemps(3) instead of mkstemp(3) for opening named temporary files Allows 'got commit' to use a ".diff" suffix for temporary diff files. ok op@

let callers of got_pack_create() configure rate-limiting of progress reporting Needed by future gotd(8), where progress reports will be sent to a network socket, rather than a local terminal.

move got_opentempfd() call out of got_pack_create() Future gotd(8) needs to run got_pack_create() in a chroot environment, so we can no longer open new temporary files inside got_pack_create(). ok op@

make got_pack_create() write to a file descriptor instead of a stdio FILE The old code required a seekable output file. This conflicts with requirements of future gotd(8), which will write pack file data to network sockets. ok op@

remove trailing whitespace; patch by Josiah Frentsos

If the first readdir() returns NULL err is uninitialized. This can't happen in practice, but llvm's scan-build doesn't know this. OK stsp

inline struct got_object_id in struct got_object_qid Saves us from doing a malloc/free call for every item on the list. ok op@

sort references by date for packing; newer commits should be processed first

factorize imsg_clear calls after imsg_flush failures imsg_clear frees and closes resources allocated as part of enqueueing imsgs so it's a no-op after reads. discussed with and ok stsp@

reuse existing deltas when creating pack files tested by thomas, naddy, and myself