// vim:set filetype=asciidoc:
[[git-packaging]]
== Packaging with *git*

Up to "`<<adv-pkg>>`", we focused on packaging operations without using https://en.wikipedia.org/wiki/Git[Git] or any other https://en.wikipedia.org/wiki/Version_control[VCS].  These traditional packaging operations were based on the tarball released by the upstream as mentioned in "`<<historical-perspective>>`".

Currently, the *git*(1) command is the de-facto platform for the VCS tool and is the essential part of both upstream development and Debian packaging activities. (See Debian wiki "`https://wiki.debian.org/GitPackagingSurvey[Debian git packaging maintainer branch formats and workflows]`" for existing VCS workflows.)

NOTE: Since the non-native Debian source package uses "`**diff -u**`" as its backend technology for the maintainer modification, it can't represent modification involving symlink, file permissions, nor binary data (https://lists.debian.org/debian-devel/2022/03/msg00124.html[March 2022 discussion on debian-devel@l.d.o]).  Please avoid making such maintainer modifications even though these can be recorded in the Git repository.

Since VCS workflows are complicated topic and there are many practice styles, I only touch on some key points with minimal information, here.

https://salsa.debian.org[Salsa] is the remote Git repository service with associated tools.  It offers the collaboration platform for Debian packaging activities using a custom https://en.wikipedia.org/wiki/GitLab[GitLab] application instance. See:

* "`<<salsa-repo>>`"
* "`<<salsa-account>>`"
* "`<<salsa-ci>>`"

There are 2 styles of branch names for the Git repository used for the packaging.  See "`<<branch-names>>`".

There are 2 main usage styles for the Git repository for the packaging. See:

* "`<<git-patch-unapplied>>`"
* "`<<git-patch-applied>>`"

There are 2 notable Debian packaging tools for the Git repository for the packaging.

* *gbp*(1) and its subcommands:
** This is a tool designed to work with "`<<git-patch-unapplied>>`".
** See "`<<gbp>>`".
* *dgit*(1) and its subcommands:
** This is a tool designed to work with both "`<<git-patch-applied>>`" and "`<<git-patch-unapplied>>`".
** This contains a tool to upload Debian packages using the *dgit* server.
** See "`<<dgit>>`".

// Sean Whitton: middleground where the git workflows don't require staying
// with 1.0? Even if that means switching to 3.0 (quilt) using the
// single-debian-patch approach?
//   https://lists.debian.org/debian-devel/2022/03/msg00122.html
//
// Ian Jackson
// 5. Why is native sometimes superior to any quilt or diff format
//
// We rely on diff(1) to represent changes to source trees andatch(1)
// to apply them.  Not every change is representable by diff.
// ...
// Currently, I think diff cannot represent changes to symlinks.
// git can store symlinks and represent their targets, and changes to
// their targets.
//  https://lists.debian.org/debian-devel/2022/03/msg00124.html
//
// My old note
//  https://osamuaoki.github.io/en/2019/02/11/dgit-workfow01/
//
// My current thought for VCS usage for Debian packaging
// * No direct binary file modification as git commit
// * No direct change between symlink and file as git commit
// As long as we address these situation via customizing d/rules,
// then "3.0 (quilt)" package is doable choice

// https://browse.dgit.debian.org/
// https://wiki.debian.org/DgitFAQ
// https://wiki.debian.org/GitPackagingSurvey
// https://wiki.debian.org/GitPackagingSurvey/git-debrebase
// https://wiki.debian.org/GitPackagingSurvey/unapplied
// https://www.chiark.greenend.org.uk/~ijackson/2015/debconf-dgit-talk/slides.pdf
// https://wiki.debian.org/DebianEvents/gb/2023/MiniDebConfCambridge/Jackson?action=AttachFile&do=get&target=talk.txt
// https://wiki.debian.org/DebianEvents/gb/2023/MiniDebConfCambridge/Jackson?action=AttachFile&do=get&target=slides.pdf

// https://wiki.debian.org/AdvancedBuildingTips
// https://wiki.debian.org/Services
// DD-list

[[salsa-repo]]
=== Salsa repository

It is highly desirable to host Debian source code package on https://salsa.debian.org[Salsa].  Over 90% of all Debian source code packages are hosted on https://salsa.debian.org[Salsa]. footnote:[Use of git.debian.org or alioth.debian.org are deprecated now.]

The exact VCS repository hosting an existing Debian source code package can be identified by a metadata field `Vcs-*` which can be viewed with the `apt-cache showsrc <package-name>` command.

[[salsa-account]]
=== Salsa account setup

After signing up for an account on https://salsa.debian.org[Salsa], make sure that the following pages have the same e-mail address and GPG keys you have configured to be used with Debian, as well as your SSH key:

* https://salsa.debian.org/-/profile/emails
* https://salsa.debian.org/-/user_settings/gpg_keys
* https://salsa.debian.org/-/user_settings/ssh_keys

[[salsa-ci]]
=== Salsa CI service

https://salsa.debian.org[Salsa] runs https://salsa.debian.org/salsa-ci-team/pipeline[Salsa CI] service as an instance of https://docs.gitlab.com/ee/ci/[GitLab CI] for "`<<ci>>`".

For every "`**git push**`" instances, tests which mimic tests run on the official Debian package service can be run by setting https://salsa.debian.org/salsa-ci-team/pipeline[Salsa CI] configuration file "`<<salsa-ci-yml>>`" as:

----
---
include:
  - https://salsa.debian.org/salsa-ci-team/pipeline/raw/master/recipes/debian.yml

# Customizations here
----

[[branch-names]]
=== Branch names

The Git repository for the Debian packaging should have at least 2 branches:

* *debian-branch* to hold the current Debian packaging head.
** old style: *master* (or *debian*, *main*, ...)
** https://dep-team.pages.debian.net/deps/dep14/[DEP-14] style: *debian/latest*
* *upstream-branch* to hold the upstream releases in the imported form.
** old style: *upstream*
** https://dep-team.pages.debian.net/deps/dep14/[DEP-14] style: *upstream/latest*

In this tutorial, old style branch names are used in examples for simplicity.

NOTE: This *upstream-branch* may need to be created using the tarball released by the upstream independent of the upstream Git repository since it tends to contain automatically generated files.

The upstream Git repository content can co-exit in the local Git repository used for the Debian packaging by adding its copy.  E.g.:

----
$ git remote add upstreamvcs <url-upstream-git-repo>
$ git fetch upstreamvcs master:upstream-master
----

This allows easy cherry-picking from the upstream Git repository for bug fixes.

[[git-patch-unapplied]]
=== Patch unapplied Git repository

The patch unapplied Git repository can be summarized as:

* This seems to be the traditional practice as of 2024.
* The source tree matches extracted contents by "`**dpkg-source -x --skip-patches**`" of the Debian source package.
** The upstream source is recorded in the Git repository without changes.
** The maintainer modified contents are confined within the *debian/** directory.
** Maintainer changes to the upstream source are recorded in **debian/patches/*** files for the Debian source format "`**3.0 (quilt)**`".
* This repository style is useful for all variants of traditional workflows and *gdb* based workflow:
** "`<<step-maintainer>>`" (no patch)
** "`<<diff-u-ap>>`"
*** **debian/patches/*** files can also be generated using "`**git format-patch**`", "`**git diff**`", or "`**gitk**`" from *git* commits in the through-away maintainer modification branch or from the upstream unreleased commits.
** "`<<dquilt-ap>>`" including the last "`**dquilt pop -a**`" step
** "`<<gbp-pq-ap>>`"
* Use helper scripts such as **dquilt**(1) and **gbp-pq**(1) to manage data in **debian/patches/*** files.
** Add *.pc* line to the *~/.gitignore* file if *dquilt* is used.
** Add *unapply-patches* and *abort-on-upstream-changes* lines in the *debian/source/local-options* file.
* Use "`*dpkg-source -b*`" to build the Debian source package.
* Use **dput**(1) to upload the Debian source package.
** Use "`**dgit --gbp push-source**`" or "`**dgit --gbp push**`" instead to upload the Debian package via the *dgit* server (see "`*dgit-maint-gbp*(7)`").

NOTE: The *debian/source/local-options* and *debian/source/local-patch-header* files are meant to be recorded by the *git* command.  These aren't included in the Debian source package.

[[git-patch-applied]]
=== Patch applied Git repository

The patch applied Git repository can be summarized as:

* The source tree matches extracted contents by "`**dpkg-source -x**`" of the Debian source package.
** The source tree is buildable and the same as what NMU maintainers see.
** The source is recorded in the Git repository with maintainer changes including the **debian/** directory.
** Maintainer changes to the upstream source are also recorded in **debian/patches/*** files for the Debian source format "`**3.0 (quilt)**`".

Use one of workflow styles:

* **dgit-maint-merge**(7) workflow.
** Use this if you don't intend to record topic patches in the Debian source package.
** Good enough for packages only with trivial modifications to the upstream.
** Only choice for packages with intertwined modification histories to the upstream
** Add *auto-commit* and *single-debian-patch* lines in the *debian/source/local-options* file
** Use "`**git checkout upstream; git pull**`" to pull the new upstream commit and use "`*git checkout master ; git merge <new-version-tag>*`" to merge it to the *master* branch.
** Use "`**dpkg-source -b**`" to build the Debian source package.
** Use "`**dgit push-source**`" or "`*dgit push*`" for uploading the Debian package via the *dgit* server.
** See "`<<auto-commit-ap>>`" for example.
* **dgit-maint-debrebase**(7) workflow.
** Use this if you wish to commit maintainer changes to the patch applied Git repository with the same granularity as patches of "`<<gbp-pq-ap>>`".
** Good for packages with multiple sequenced modifications to the upstream.
** Use "`**dgit build-source**`" to build the Debian source package.
** Use "`**dgit push-source**`" or "`*dgit push*`" for uploading the Debian package via the *dgit* server.
** Details of this workflow are beyond the scope of this tutorial document.  See "`<<dgit-maint-debrebase>>`" for more.

[[gbp]]
=== Note on *gbp*

The *gbp* command is provided by the *git-buildpackage* package.

* This command is designed to manage contents of "`<<git-patch-unapplied>>`" efficiently.
* Use "`**gbp import-orig**`" to import the new upstream tar to the git repository.
** The "`**--pristine-tar**`" option for the "`*git import-orig*`" command enables storing the upstream tarball in the same git repository.
** The "`**--uscan**`" option as the last argument of the "`*gbp import-orig*`" command enables downloading and committing the new upstream tarball into the git repository.
* Use "`**gbp import-dsc**`" to import the previous Debian source package to the git repository.
* Use "`**gbp dch**`" to generate the Debian changelog from the git commit messages.
* Use "`**gbp buildpackage**`" to build the Debian binary package from the git repository.
** The *sbuild* package can be used as its clean chroot build backend either by configuration or adding "`*--git-builder='sbuild -A -s --source-only-changes -v -d unstable'*`"
* Use "`**gbp pull**`" to update the *debian*, *upstream* and *pristine-tar* branches safely from the remote repository.
* Use "`**gbp pq**`" to manage quilt patches without using *dquilt* command.
* Use "`**gbp clone** __REPOSITORY_URL__`" to clone and set up tracking branches for *debian*, *upstream* and *pristine-tar*.

Package history management with the *git-buildpackage* package is becoming the standard practice for many Debian maintainers.  See more at:

* "`https://honk.sigxcpu.org/projects/git-buildpackage/manual-html/gbp.html[Building Debian Packages with git-buildpackage]`"
* "`https://raphaelhertzog.com/2010/11/18/4-tips-to-maintain-a-3-0-quilt-debian-source-package-in-a-vcs/[4 tips to maintain a “3.0 (quilt)” Debian source package in a VCS]`"
* The *systemd* packaging practice documentation on "`https://salsa.debian.org/systemd-team/systemd/-/blob/debian/master/debian/README.source[Building from source]`"
* The workflow mentioned in **dgit-maint-gbp**(7) which enables to use this *gbp* with *dgit*

[[dgit]]
=== Note on *dgit*

The *dgit* command is provided by the *dgit* package.

* This command is designed to manage contents of "`<<git-patch-applied>>`" efficiently.
** This enables to access the Debian package repository as if it is a *git* remote repository.
* This command supports uploading Debian packages using the *dgit* server from both "`<<git-patch-unapplied>>`" and "`<<git-patch-applied>>`".

The new *dgit* package offers commands interact with the Debian repository as if it was a git repository. It does not replace *gbp-buildpackage* and both can be used at the same time. Using plain *gbp-buildpackage* is recommended for developers who want to run git push/pull on Salsa and use things such as Salsa CI or Merge Requests on Salsa.

For more details see the extensive guides:

* **dgit-maint-gbp**(7) -- for the Debian source format "`**3.0 (quilt)**`" package with its Debian Git repository which is kept usable also for people using **gbp-buildpackage**(1) using "`<<git-patch-unapplied>>`".
* **dgit-maint-merge**(7) -- for the Debian source format "`**3.0 (quilt)**`" package with its changes flowing both ways between the upstream Git repository and the Debian Git repository which are tightly coupled using "`<<git-patch-applied>>`".
* **dgit-maint-debrebase**(7) -- for the Debian source format "`**3.0 (quilt)**`" package with its changes flowing mostly one way from the upstream Git repository to the Debian Git repository using "`<<git-patch-applied>>`".
* **dgit-maint-native**(7) -- for the Debian source format "`**3.0 (native)**`" package in the Debian Git repository. (No maintainer changes)

The **dgit**(1) command can push the easy-to-trace change history to the https://browse.dgit.debian.org/[https://browse.dgit.debian.org/] site and can upload Debian package to the Debian repository properly without using **dput**(1).

The concept around *dgit* is beyond this tutorial document.  Please start reading relevant information:

* "`https://www.chiark.greenend.org.uk/~ijackson/2015/debconf-dgit-talk/slides.pdf[dgit: use the Debian archive as a git remote (2015)]`"
* "`https://wiki.debian.org/DebianEvents/gb/2023/MiniDebConfCambridge/Jackson?action=AttachFile&do=get&target=slides.pdf[tag2upload (2023)]`"


[[gbp-pq-ap]]
=== Patch by "`*gbp-pq*`" approach

For "`<<git-patch-unapplied>>`", you can generate **debian/patches/*** files using the **gbp-pq**(1) command from *git* commits in the through-away *patch-queue* branch.

Unlike *dquilt* which offers similar functionality as seen "`<<dquilt-ap>>`" and "`<<dquilt-manage>>`", *gbp-pq* doesn't use *.pc/** files to track patch state, but instead *gbp-pq* utilizes temporary branches in git.

[[gbp-manage]]
=== Manage patch queue with *gbp-pq*

You can add, drop, and refresh **debian/patches/*** files with *gbp-pq* to manage patch queue.

If the package is managed in "`<<git-patch-unapplied>>`" using the *git-buildpackage* package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using *gbp pq*.

* **Add** a new patch recording the upstream source modification on the file __buggy_file__ as:
+
--
----
 $ git checkout master
 $ gbp pq import
gbp:info: ... imported on 'patch-queue/master
 $ vim buggy_file
  ...
 $ git add buggy_file
 $ git commit
 $ gbp pq export
gbp:info: On 'patch-queue/master', switching to 'master'
gbp:info: Generating patches from git (master..patch-queue/master)
 $ git add debian/patches/*
 $ dch -i
 $ git commit -a -m "Closes: #<bug_number>"
 $ git tag debian/<version>-<rev>
----
--
* **Drop** (== disable) an existing patch
** Comment out pertinent line in *debian/patches/series*
** Erase the patch itself (optional)
* **Refresh** **debian/patches/*** files to make "`**dpkg-source -b**`" work as expected after updating a Debian package to the new upstream release.
+
--
----
 $ git checkout master
 $ gbp pq --force import # ensure patch-queue/master branch
gbp:info: ... imported on 'patch-queue/master
 $ git checkout master
 $ gbp import-orig --pristine-tar --uscan
  ...
gbp:info: Successfully imported version ?.?.? of ../packagename_?.?.?.orig.tar.gz
 $ gbp pq rebase
 ... resolve conflicts and commit to patch-queue/master branch
 $ gbp pq export
gbp:info: On 'patch-queue/master', switching to 'master'
gbp:info: Generating patches from git (master..patch-queue/master)
 $ git add debian/patches
 $ git commit -m "Update patches"
 $ dch -v <newversion>-1
 $ git commit -a -m "release <newversion>-1"
 $ git tag debian/<newversion>-1
----
--

[[debsnap]]
=== *gbp import-dscs --debsnap*

For Debian source packages named "`__<source-package>__`" recorded in the http://snapshot.debian.org/[snapshot.debian.org] archive, an initial git repository managed in "`<<git-patch-unapplied>>`" with all of the Debian version history can be generated as follows.

----
$ gbp import-dscs --debsnap --pristine-tar <source-package>
----

[[dgit-maint-debrebase]]
=== Note on **dgit-maint-debrebase** workflow

Here are some hints around **dgit-maint-debrebase**(7). footnote:[I may be incorrect, here.]

* Use "`**dgit setup-new-tree**`" to prepare the local *git* working repository.
* The first maintainer modification commit should contain files only in the *debian/* directory excluding files in the *debian/patches* directory.
* **debian/patches/*** files are generated from the maintainer modification commit history using the "`**dgit quilt-fixup**`" command automatically invoked from "`**dgit build**`" and "`**dgit push**`".
* Use "`**git-debrebase new-version <new-version-tag>**`" to rebase the maintainer modification commit history with automatically updated *debian/changelog*.
* Use "`**git-debrebase conclude**`" to make a new pseudomerge (== "`**git merge -s ours**`") to record Debian package with clean ff-history.

See **dgit-maint-debrebase**(7), **dgit**(1) and **git-debrebase**(1) for more.

[[quasi-native]]
=== *Quasi-native* Debian packaging

The *quasi-native* packaging scheme packages a source without the real upstream tarball using the *non-native* package format.

TIP: Some people promote this *quasi-native* packaging scheme even for programs written only for Debian since it helps to ease communication with the downstream distros such as Ubuntu for bug fixes etc.

This *quasi-native* packaging scheme involves 2 preparation steps:

* Organize its source tree almost like *native* Debian package (see "`<<native>>`") with *debian/** files with a few exceptions:
** Make *debian/source/format* to contain "`**3.0 (quilt)**`" instead of "`**3.0 (native)**`" .
** Make *debian/changelog* to contain __version__**-**__revision__ instead of __version__ .
* Generate missing upstream tarball preferably without *debian/** files.
** For Debian source format "`**3.0 (quilt)**`", removal of files under *debian/* directory is an optional action.

The rest is the same as the *non-native* packaging workflow as written in "`<<workflow>>`".

Although this can be done in many ways ("`<<dt-option>>`"), you can use the Git repository and "`**git deborig**`" as:

----
 $ cd /path/to/<dirname>
 $ dch -r
  ... set its <version>-<revision>, e.g., 1.0-1
 $ git tag -s debian/1.0-1
 $ git rm -rf debian
 $ git tag -s upstream/1.0
 $ git commit -m upstream/1.0
 $ git reset --hard HEAD^
 $ git deborig
 $ sbuild
----

// KEEP HERE TO ADDRESS DGIT XXX FIXME XXX

// [[dgit-init]]
// === *dgit* (initial)
//
// TODO: XXX FIXME XXX
//
// If the package is managed in "`<<git-patch-applied>>`" using the **dgit**(1) package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using **git-debrebase**(1).
//
// // git-debrebase convert-from-unapplied
//
// .New Debian revision with the *git debrebase* command
// ----
//  $ git clone -o upstream -b upstream UPSTREAM_GIT_REPO
//  $ git checkout -b master
//  $  ... add debian/* files (no debian/patches)
//  $ vim buggy_file
//   ...
//  $ git add buggy_file
//  $ dch -i -r
//  $ git commit -a -m "Closes: #<bug_number>"
//  $ git debrebase
//  $ dgit quilt-fixup
//  $ git debrebase conclude
//  $ dgit build-source
// ----
//
// [[dgit-revise]]
// === *dgit* (revision)
//
// TODO: XXX FIXME XXX
//
// If the package is managed in "`<<git-patch-applied>>`" using the **dgit**(1) package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using **git-debrebase**(1).
//
// // git-debrebase convert-from-unapplied
//
// .New Debian revision with the *git debrebase* command
// ----
//  $ git clone -o upstream -b upstream UPSTREAM_GIT_REPO
//  $ git checkout -b master
//  $  ... add debian/* files (no debian/patches)
//  $ vim buggy_file
//   ...
//  $ git add buggy_file
//  $ dch -i -r
//  $ git commit -a -m "Closes: #<bug_number>"
//  $ git debrebase
//  $ dgit quilt-fixup
//  $ git debrebase conclude
//  $ dgit build-source
// ----
//
// [[dgit-new-upstream]]
// === *dgit* (new upstream)
//
// TODO: XXX FIXME XXX
//
// If the package is managed in "`<<git-patch-applied>>`" using the **dgit**(1) package, you can revise the upstream source to fix bug as the maintainer and release a new Debian revision using **git-debrebase**(1).
//
// You can automatically update to the new upstream source with the "`*gbp import-orig --pristine-tar*`" command from the *git-buildpackage* package.  It requires having the old Debian source in the git repository and the new upstream tarball.
// // git-debrebase convert-from-unapplied
//
// .New Debian revision with the *git debrebase* command
// ----
//  $ git checkout master
//  $ git debrebase
//  $ vim buggy_file
//   ...
//  $ git add buggy_file
//  $ dch -i -r
//  $ git commit -a -m "Closes: #<bug_number>"
//  $ dgit quilt-fixup
//  $ git debrebase conclude
//  $ dgit build-source
// ----
//
