Formed in 2009, the Archive Team (not to be confused with the archive.org Archive-It Team) is a rogue archivist collective dedicated to saving copies of rapidly dying or deleted websites for the sake of history and digital heritage. The group is 100% composed of volunteers and interested parties, and has expanded into a large amount of related projects for saving online and digital history.
Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upRFC: support streaming a block index' content into a char device #227
Conversation
A reasonable exemplary usecase for this is updating a UBI volume (its interface to userspace is a char device): let a parent process open the device file and prepare the resulting file descriptor for updating the UBI volume, then fork 'casync extract <BLOB_INDEX> -' with STDOUT being redirected to that file descriptor. Without this change the only working alternative today is to fork 'casync mkdev <BLOB_INDEX>' and then to actively pipe the resulting block device's content into the UBI volume's prepared character device which is uneconomic as the whole data has to be shuffled around once by casync and a second time by the parent process. Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
OnkelUlla
added a commit
to OnkelUlla/rauc
that referenced
this pull request
Oct 24, 2019
…bx" images Note: for this to work successfully the utilized casync has to be augmented by (the currently unmerged) pull request [1]. [1] systemd/casync#227 Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
OnkelUlla
added a commit
to OnkelUlla/rauc
that referenced
this pull request
Oct 25, 2019
…bx" images Note: for this to work successfully the accompanying casync has to be augmented by (the currently unmerged) casync pull request [1]. [1] systemd/casync#227 Signed-off-by: Ulrich Ölmann <u.oelmann@pengutronix.de>
jluebbe
added a commit
to rauc/rauc
that referenced
this pull request
Feb 10, 2020
Introduce casync based bundles to the update of UBI volumes This still needs the systemd/casync#227 to work, but should not break existing cases.
|
@poettering Is there something we could do to make this easier to merge? Is the general approach acceptable? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Within RAUC (an updating solution for embedded systems) we want to apply
casyncto the usecase of updating UBI volumes whose interface to userspace consists of char devices: to do so the only working solution today without intermediate extraction to a temporary regular file would be to spawncasync mkdev <BLOB_INDEX>and then to copy the whole contents of the attached/dev/nbdXover to the UBI volume's char device/dev/ubiY_Z. This is inefficient as the data has to be moved around bycasyncfirst and a second time by the copying mechanism. It would definitively be resource saving ifcasync extract <BLOB_INDEX> /dev/ubiY_Zworks which is what the pull request's commit finally permits.As
casyncchecks the output's file descriptor for belonging to a regular file or a block device several times in the code I am uncertain if there are implicit assumptions incasync'sdecoder that I am not aware of and that are breaking down now (that's why I put the RFC in front of the pull request's title). So the question is if my patch is valid at all?Although what this pull request changes is enough for our usecase it would definitely be appreciated if
casync extract <BLOB_INDEX>could cope with pipes as output, too. Having that feature would allow to manually do things like# casync extract <BLOB_INDEX> | ubiupdatevol /dev/ubiY_Z --size=<bytes> -on the commandline. It should be straight forward to extend this patch with
S_ISFIFO(…)correspondingly.