dulwich.object_store module

Git object store interfaces and implementation.

class dulwich.object_store.BaseObjectStore

Bases: object

Object store interface.

add_object(obj)

Add a single object to this object store.

add_objects(objects, progress=None)

Add a set of objects to this object store.

Parameters

objects – Iterable over a list of (object, path) tuples

add_pack() → Tuple[BytesIO, Callable[[], None], Callable[[], None]]

Add a new pack to this object store.

add_pack_data(count, pack_data, progress=None)

Add pack data to this object store.

Parameters

  • count – Number of items to add

  • pack_data – Iterator over pack data tuples

close()

Close any files opened by this object store.

contains_loose(sha)

Check if a particular object is present by SHA1 and is loose.

contains_packed(sha)

Check if a particular object is present by SHA1 and is packed.

determine_wants_all(refs: Dict[bytes, bytes], depth: Optional[int] = None) → List[bytes]
find_common_revisions(graphwalker)

Find which revisions this store has in common using graphwalker.

Parameters

graphwalker – A graphwalker object.

Returns: List of SHAs that are in common

find_missing_objects(haves, wants, shallow=None, progress=None, get_tagged=None, get_parents=<function BaseObjectStore.<lambda>>, depth=None)

Find the missing objects required for a set of revisions.

Parameters

  • haves – Iterable over SHAs already in common.

  • wants – Iterable over SHAs of objects to fetch.

  • shallow – Set of shallow commit SHA1s to skip

  • progress – Simple progress function that will be called with updated progress strings.

  • get_tagged – Function that returns a dict of pointed-to sha -> tag sha for including tags.

  • get_parents – Optional function for getting the parents of a commit.

Returns: Iterator over (sha, path) pairs.

generate_pack_contents(have, want, shallow=None, progress=None)

Iterate over the contents of a pack file.

Parameters

  • have – List of SHA1s of objects that should not be sent

  • want – List of SHA1s of objects that should be sent

  • shallow – Set of shallow commit SHA1s to skip

  • progress – Optional progress reporting method

generate_pack_data(have, want, shallow=None, progress=None, ofs_delta=True)

Generate pack data objects for a set of wants/haves.

Parameters

  • have – List of SHA1s of objects that should not be sent

  • want – List of SHA1s of objects that should be sent

  • shallow – Set of shallow commit SHA1s to skip

  • ofs_delta – Whether OFS deltas can be included

  • progress – Optional progress reporting method

get_raw(name)

Obtain the raw text for an object.

Parameters

name – sha for the object.

Returns: tuple with numeric type and object contents.

iter_shas(shas)

Iterate over the objects for the specified shas.

Parameters

shas – Iterable object with SHAs

Returns: Object iterator

iter_tree_contents(tree_id, include_trees=False)

Iterate the contents of a tree and all subtrees.

Iteration is depth-first pre-order, as in e.g. os.walk.

Parameters

  • tree_id – SHA1 of the tree.

  • include_trees – If True, include tree objects in the iteration.

Returns: Iterator over TreeEntry namedtuples for all the objects in a

tree.

property packs

Iterable of pack objects.

peel_sha(sha)

Peel all tags from a SHA.

Parameters

sha – The object SHA to peel.

Returns: The fully-peeled SHA1 of a tag object, after peeling all

intermediate tags; if the original ref does not point to a tag, this will equal the original SHA1.

tree_changes(source, target, want_unchanged=False, include_trees=False, change_type_same=False, rename_detector=None)

Find the differences between the contents of two trees

Parameters

  • source – SHA1 of the source tree

  • target – SHA1 of the target tree

  • want_unchanged – Whether unchanged files should be reported

  • include_trees – Whether to include trees

  • change_type_same – Whether to report files changing type in the same entry.

Returns: Iterator over tuples with

(oldpath, newpath), (oldmode, newmode), (oldsha, newsha)

class dulwich.object_store.BucketBasedObjectStore(pack_compression_level=-1)

Bases: PackBasedObjectStore

Object store implementation that uses a bucket store like S3 as backend.

add_pack()

Add a new pack to this object store.

Returns: Fileobject to write to, a commit function to

call when the pack is finished and an abort function.

class dulwich.object_store.DiskObjectStore(path, loose_compression_level=-1, pack_compression_level=-1)

Bases: PackBasedObjectStore

Git-style object store that exists on disk.

Open an object store.

Parameters

  • path – Path of the object store.

  • loose_compression_level – zlib compression level for loose objects

  • pack_compression_level – zlib compression level for pack objects

add_alternate_path(path)

Add an alternate path to this object store.

add_object(obj)

Add a single object to this object store.

Parameters

obj – Object to add

add_pack()

Add a new pack to this object store.

Returns: Fileobject to write to, a commit function to

call when the pack is finished and an abort function.

add_thin_pack(read_all, read_some)

Add a new thin pack to this object store.

Thin packs are packs that contain deltas with parents that exist outside the pack. They should never be placed in the object store directly, and always indexed and completed as they are copied.

Parameters

  • read_all – Read function that blocks until the number of requested bytes are read.

  • read_some – Read function that returns at least one byte, but may not return the number of bytes requested.

Returns: A Pack object pointing at the now-completed thin pack in the

objects/pack directory.

property alternates
classmethod from_config(path, config)
classmethod init(path)
move_in_pack(path)

Move a specific file containing a pack into the pack directory.

Note: The file should be on the same file system as the

packs directory.

Parameters

path – Path to the pack file.

class dulwich.object_store.MemoryObjectStore

Bases: BaseObjectStore

Object store that keeps all objects in memory.

add_object(obj)

Add a single object to this object store.

add_objects(objects, progress=None)

Add a set of objects to this object store.

Parameters

objects – Iterable over a list of (object, path) tuples

add_pack()

Add a new pack to this object store.

Because this object store doesn’t support packs, we extract and add the individual objects.

Returns: Fileobject to write to and a commit function to

call when the pack is finished.

add_thin_pack(read_all, read_some)

Add a new thin pack to this object store.

Thin packs are packs that contain deltas with parents that exist outside the pack. Because this object store doesn’t support packs, we extract and add the individual objects.

Parameters

  • read_all – Read function that blocks until the number of requested bytes are read.

  • read_some – Read function that returns at least one byte, but may not return the number of bytes requested.

contains_loose(sha)

Check if a particular object is present by SHA1 and is loose.

contains_packed(sha)

Check if a particular object is present by SHA1 and is packed.

get_raw(name: bytes)

Obtain the raw text for an object.

Parameters

name – sha for the object.

Returns: tuple with numeric type and object contents.

property packs

List with pack objects.

class dulwich.object_store.MissingObjectFinder(object_store, haves, wants, shallow=None, progress=None, get_tagged=None, get_parents=<function MissingObjectFinder.<lambda>>)

Bases: object

Find the objects missing from another object store.

Parameters

  • object_store – Object store containing at least all objects to be sent

  • haves – SHA1s of commits not to send (already present in target)

  • wants – SHA1s of commits to send

  • progress – Optional function to report progress to.

  • get_tagged – Function that returns a dict of pointed-to sha -> tag sha for including tags.

  • get_parents – Optional function for getting the parents of a commit.

  • tagged – dict of pointed-to sha -> tag sha for including tags

add_todo(entries)
next()
class dulwich.object_store.ObjectIterator

Bases: object

Interface for iterating over objects.

iterobjects()
class dulwich.object_store.ObjectStoreGraphWalker(local_heads, get_parents, shallow=None)

Bases: object

Graph walker that finds what commits are missing from an object store.

heads

Revisions without descendants in the local repo

get_parents

Function to retrieve parents in the local repo

Create a new instance.

Parameters

  • local_heads – Heads to start search with

  • get_parents – Function for finding the parents of a SHA1.

ack(sha)

Ack that a revision and its ancestors are present in the source.

next()

Iterate over ancestors of heads in the target.

class dulwich.object_store.ObjectStoreIterator(store, sha_iter)

Bases: ObjectIterator

ObjectIterator that works on top of an ObjectStore.

Create a new ObjectIterator.

Parameters

  • store – Object store to retrieve from

  • sha_iter – Iterator over (sha, path) tuples

iterobjects()

Iterate over just the objects.

itershas()

Iterate over the SHAs.

class dulwich.object_store.OverlayObjectStore(bases, add_store=None)

Bases: BaseObjectStore

Object store that can overlay multiple object stores.

add_object(object)

Add a single object to this object store.

add_objects(objects, progress=None)

Add a set of objects to this object store.

Parameters

objects – Iterable over a list of (object, path) tuples

contains_loose(sha)

Check if a particular object is present by SHA1 and is loose.

contains_packed(sha)

Check if a particular object is present by SHA1 and is packed.

get_raw(sha_id)

Obtain the raw text for an object.

Parameters

name – sha for the object.

Returns: tuple with numeric type and object contents.

property packs

Iterable of pack objects.

class dulwich.object_store.PackBasedObjectStore(pack_compression_level=-1)

Bases: BaseObjectStore

add_objects(objects, progress=None)

Add a set of objects to this object store.

Parameters

objects – Iterable over (object, path) tuples, should support __len__.

Returns: Pack object of the objects written.

property alternates
close()

Close any files opened by this object store.

contains_loose(sha)

Check if a particular object is present by SHA1 and is loose.

This does not check alternates.

contains_packed(sha)

Check if a particular object is present by SHA1 and is packed.

This does not check alternates.

get_raw(name)

Obtain the raw fulltext for an object.

Parameters

name – sha for the object.

Returns: tuple with numeric type and object contents.

pack_loose_objects()

Pack loose objects.

Returns: Number of objects packed

property packs

List with pack objects.

repack()

Repack the packs in this repository.

Note that this implementation is fairly naive and currently keeps all objects in memory while it repacks.

dulwich.object_store.commit_tree_changes(object_store, tree, changes)

Commit a specified set of changes to a tree structure.

This will apply a set of changes on top of an existing tree, storing new objects in object_store.

changes are a list of tuples with (path, mode, object_sha). Paths can be both blobs and trees. See the mode and object sha to None deletes the path.

This method works especially well if there are only a small number of changes to a big tree. For a large number of changes to a large tree, use e.g. commit_tree.

Parameters

  • object_store – Object store to store new objects in and retrieve old ones from.

  • tree – Original tree root

  • changes – changes to apply

Returns: New tree root object

dulwich.object_store.read_packs_file(f)

Yield the packs listed in a packs file.

dulwich.object_store.tree_lookup_path(lookup_obj, root_sha, path)

Look up an object in a Git tree.

Parameters

  • lookup_obj – Callback for retrieving object by SHA1

  • root_sha – SHA1 of the root tree

  • path – Path to lookup

Returns: A tuple of (mode, SHA) of the resulting path.