summaryrefslogtreecommitdiff
path: root/src/Propellor/Property/DiskImage.hs
AgeCommit message (Collapse)Author
2018-01-06changes to allow GPT BIOS boot partitionsJoey Hess
* Parted: Allow partitions to have no filesystem, for eg, GPT BIOS boot partitions. (API change) * Added rawPartition to PartSpec, for specifying partitions with no filesystem. * Added BiosGrubFlag to PartFlag. Note that man parted does not list the "bios_boot" flag, but I found it in its html documentation. Other flags may also be missing. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2017-12-21Grub.boots, Grub.bootsMounted: Pass --target to grub-install.Joey Hess
This is to support eg, coreboot. The GrubTarget passed to Grub.installed is introspected to determine --target. If multiple grubs are installed, it currently doesn't pass any --target. Might make more sense to run grub-install repeatedly, but I don't know if that case is sane at all. The Xen -> "x86_64-xen" mapping is kind of arbitrarily chosen since there's a i386-xen available too. I don't know when that case would be used in any case though; chainPVGrub uses installed Xen, but it does not run grub-install. If this does become a problem, would probably need to split it into Xen64 and Xen32. Renamed BIOS to GrubTarget in passing to match grub's terminology; BIOS was kind of a joke term for this in propellor. This commit was sponsored by Francois Marier on Patreon.
2017-12-20say when resizing/creating disk image fileJoey Hess
That can take quite a while, so let the user know why propellor has stalled.
2017-12-20two diskimage edge case fixesJoey Hess
* DiskImage: Fix rsync crash when a mount point does not exist in the chroot. * Fix bug in unmountBelow that caused unmounting of nested mounts to fail. This commit was sponsored by Jack Hill on Patreon.
2017-12-20disk partition alignmentJoey Hess
Cheap flash drives need partitions aligned to 4 MiB in order to not be slow (and to avoid extra writes). <https://lwn.net/Articles/428584/> And at least 1 MiB alignment is generally a good idea, and most people seem to think 4 MiB is for all drives. I noticed that Parted.partitioned does not do that; the first partition started at an offset of 1 MB, and subsequent partitions from where it ends. (The 1 MB offset came from the PartedVal PartSize instance, and note that it was not 1 MiB.) * Parted: Add an Alignment parameter. (API change) A good default to use is safeAlignment, which is 4MiB, well suited for inexpensive flash drives, and fine for other disks too. Previously, a very non-optimial 1MB (not 1MiB) alignment had been used. * DiskImage: Use safeAlignment. It didn't seem worth making the alignment configurable here. Alignment is implemented by offsetting the first partition's start position so it's aligned (making sure to leave room for the partition table). Each partition is then extended as needed so the next partition will start properly aligned. Note that parted rejects partition tables that don't fit in cylinder bounderies. Before, propellor let parted deal with the fine details of layout, so that was not a problem. Now it's possible to set some wacky Alignment not divisible by 512, or use Byte sizes for partitions and create a partition table that parted rejects. But, using safeAlignment and MegaBytes should always be safe. Also, this fixes a rounding bug in Parted.calcPartTable. It was rounding up to the nearest MegaByte when allocating remaining disk space, so returned partition table that was actually larger than the disk size. This commit was sponsored by an anonymous bitcoiner.
2017-11-19add commaJoey Hess
2017-11-19partition table in InfoJoey Hess
Diskimage.imageBuiltFor: New property to build a disk image for a Host, using partition table information configured via the new properties hasPartitionTableType, hasPartition and adjustPartition. This lets Machine properties include eg /boot partitions that are known to be needed by the bootloader, and the user can adjust those partitions and add others. This commit was sponsored by Brock Spratlen on Patreon.
2017-11-18reorganizedJoey Hess
2017-11-17Service: Avoid starting services when noServices is used.Joey Hess
Reconsidered making services never run inside chroots, that seemed too potentially limiting. Using Info rather than checking policy-rc.d because it will also work outside of debian, but more because policy-rc.d has an extremely complicated interface and I didn't want to deal with it. This commit was sponsored by Jochen Bartl on Patreon.
2017-11-17propellor spinJoey Hess
2017-11-17update initramfs and flash-kernel during disk image finalizationJoey Hess
flashKernelMounted is slightly cargo culted from Grub.bootsMounted, could be refactored. This commit was sponsored by Thom May on Patreon.
2017-11-17clean up qemu emulation binary when finalizing disk imageJoey Hess
This commit was sponsored by Denis Dzyubenko on Patreon.
2017-11-16propellor spinJoey Hess
2017-11-16propellor spinJoey Hess
2017-11-16Uboot: New module.Joey Hess
Installing u-boot to the boot sector is not needed by some boards (my CubieTruck boots without it), but may be by others. Tricky part was making u-boot be written to a disk image when building one. This commit was sponsored by Jake Vosloo on Patreon.
2017-11-16flash-kernel supportJoey Hess
Can be used to create disk images for arm boards using flash-kernel. This commit was sponsored by Ewen McNeill.
2017-08-24DiskImage type classJoey Hess
* DiskImage: Made a DiskImage type class, so that different disk image formats can be implemented. The properties in this module can generate any type that is a member of DiskImage. (API change) (To convert existing configs, convert the filename of the disk image to RawDiskImage filename.) * Removed DiskImage.vmdkBuiltFor property. (API change) Instead, use VirtualBoxPointer in the property that creates the disk image. This commit was sponsored by Jack Hill on Patreon.
2017-07-28propellor spinJoey Hess
2017-07-27DiskImage: Fix strictness bug in .parttable read/write sequence.Joey Hess
2017-07-25VBoxManage errors out if file already exists, so delete the old oneJoey Hess
2017-07-25VBoxManage creates vmdk with locked down read modeJoey Hess
2017-07-25propellor spinJoey Hess
2017-07-25new propertiesJoey Hess
* Added Rsync.installed property. * Added DiskImage.vmdkBuilt property which is useful for booting a disk image in VirtualBox.
2017-07-25remove parttablefile on reversionJoey Hess
2017-07-25DiskImage: Avoid re-partitioning disk image unncessarily, for a large speedup.Joey Hess
This commit was sponsored by Anthony DeRobertis on Patreon.
2017-07-21calcPartTable using PartSpec DSLJoey Hess
* Generalized the PartSpec DSL, so it can be used for both disk image partitioning, and disk device partitioning, with different partition sizing methods as appropriate for the different uses. (minor API change) * Propellor.Property.Parted: Added calcPartTable function which uses PartSpec DiskPart, and a useDiskSpace combinator. This commit was sponsored by Thomas Hochstein on Patreon.
2017-07-21improve commentJoey Hess
2017-07-05fix chroot info accessJoey Hess
2017-07-05DiskImage: Removed grubBootedJoey Hess
Properties that used to need it as a parameter now look at Info about the bootloader that is installed in the chroot that the disk image is created from. (API change) This is a simplication, and avoids the user needing to repeat themselves in the propellor config, thus avoiding mistakes. When no boot loader is installed, or multiple different ones are, disk image creation will fail, which seems reasonable. This commit was sponsored by Jake Vosloo on Patreon.
2017-07-05Image.grubBooted no longer takes a BIOS parameterJoey Hess
* DiskImage.grubBooted no longer takes a BIOS parameter, and no longer implicitly adds Grub.installed to the properties of the disk image. If you used DiskImage.grubBooted, you'll need to update your propellor configuration, removing the BIOS parameter from grubBooted and adding a Grub.installed property to the disk image, eg: & Grub.installed PC (API change) * Grub.installed: Avoid running update-grub when used in a chroot, since it will get confused. * DiskImage.Finalization: Simplified this type since it does not need to be used to install packages anymore. (API change) The advantage of doing this comes when using hostChroot with imageBuilt, since the Host then has its Grub.installed property explicitly listed so propellor knows about it when otherwise deploying that host. Also, it simplifies the quite complex imageBuilt parameters. This commit was sponsored by Ewen McNeill.
2017-07-05XFCE and applyPath propertiesJoey Hess
* Propellor.Property.XFCE added with some useful properties for the desktop environment. * Added File.applyPath property. This commit was sponsored by Riku Voipio.
2017-07-04Diskimage.imageExists: Align disk image size to multiple of 4096 sector sizeJoey Hess
Since some programs (such as VBoxManage convertdd) refuse to operate on disk images not aligned to a sector size. This commit was sponsored by Boyd Stephen Smith Jr. on Patreon.
2017-03-24fix diskimage finalization bugJoey Hess
* Fix bug when using setContainerProps with a chroot that prevented properties added to a chroot that way from being seen when propellor was running inside the chroot. This affected disk image creation, and possibly other things that use chroots. The problem was, propagateChrootInfo was being passed the initial version of the Chroot, but then the Chroot got more properties added, and so those were not recorded in the _chroot info. Fix was simply to make InfoPropagator be passed the Chroot as an additional parameter, so Chroot.provisioned' can pass in the final Chroot to it.
2017-03-11don't propagate DNS info from DiskImage chrootsJoey Hess
* DiskImage building properties used to propagate DNS info out from the chroot used to build the disk image to the Host. That is no longer done, since that chroot only exists as a side effect of the disk image creation and servers will not be running in it. * The IsInfo types class's propagateInfo function changed to use a PropagateInfo data type. (API change) This is particularly important when using hostChroot, since the host could well have DNS settings then. This commit was sponsored by Ole-Morten Duesund on Patreon.
2017-03-11Implemented hostChrootJoey Hess
As originally seen in my slides at Linux.Conf.Au 2017 in January. Now that it's not vaporware, it allows one Host to build a disk image that has all the properties of another Host. It was easier than I thought to implement this! As expected, Info propagation was slightly tricky. Also, I originally had a lot of machinery to try to use Info to detect infinitely nested chroot loops. But, my machinery didn't work, and when I tested it, ghc did a much better job, causing a "warning: <<loop>>" message to be output instead of such a property using infinite disk space. This commit was sponsored by Bruno BEAUFILS on Patreon.
2017-03-11reorg docsJoey Hess
2017-03-11fix exampleJoey Hess
2017-02-04replace Linux with DebianLike to compile DiskImageZihao Wang
Signed-off-by: Zihao Wang <dev@wzhd.org>
2016-06-13convert Architecture to a sumtypeFélix Sipma
TODO: remove ANDROID (used in GitAnnexBuilder) TODO: add other architectures TODO: rename ARMHF TODO: rename ARMEL (cherry picked from commit 6f36f6cade4e1d8b15c714565e223562c6573099)
2016-04-07Added Propellor.Property.Fstab, and moved the fstabbed property to there.Joey Hess
2016-03-28one moreJoey Hess
2016-03-27ported DiskImageJoey Hess
Unfortunately, DiskImage needs to add properties to the Chroot it's presented with, and the metatypes are not included in the Chroot, so it can't guarantee that the properties it's adding match the OS in the Chroot. I partially worked around this by making the properties that DiskImage adds check the OS, so they don't assume Debian. It would be nicer to parameterize the Chroot type with the metatypes of the inner OS. I worked for several hours on a patch along those lines, but it doesn't quite compile. Failed at the final hurdle :/ The patch is below for later.. --- src/Propellor/Property/Chroot.hs 2016-03-27 16:06:44.285464820 -0400 +++ /home/joey/Chroot.hs 2016-03-27 15:32:29.073416143 -0400 @@ -1,9 +1,9 @@ -{-# LANGUAGE FlexibleContexts, GADTs, DeriveDataTypeable #-} +{-# LANGUAGE FlexibleContexts, GADTs, DeriveDataTypeable, MultiParamTypeClasses, TypeSynonymInstances, FlexibleInstances, DataKinds #-} module Propellor.Property.Chroot ( debootstrapped, bootstrapped, - provisioned, + --provisioned, Chroot(..), ChrootBootstrapper(..), Debootstrapped(..), @@ -11,7 +11,7 @@ noServices, inChroot, -- * Internal use - provisioned', + --provisioned', propagateChrootInfo, propellChroot, chain, @@ -20,6 +20,7 @@ import Propellor.Base import Propellor.Container +import Propellor.Types.MetaTypes import Propellor.Types.CmdLine import Propellor.Types.Chroot import Propellor.Types.Info @@ -38,27 +39,29 @@ -- | Specification of a chroot. Normally you'll use `debootstrapped` or -- `bootstrapped` to construct a Chroot value. -data Chroot where - Chroot :: ChrootBootstrapper b => FilePath -> b -> Host -> Chroot - -instance IsContainer Chroot where - containerProperties (Chroot _ _ h) = containerProperties h - containerInfo (Chroot _ _ h) = containerInfo h - setContainerProperties (Chroot loc b h) ps = Chroot loc b (setContainerProperties h ps) +-- +-- The inner and outer type variables are the metatypes of the inside of +-- the chroot and the system it runs in. +data Chroot inner outer where + Chroot :: ChrootBootstrapper b inner outer => FilePath -> b -> Host -> (inner, outer) -> Chroot inner outer + +instance IsContainer (Chroot inner outer) where + containerProperties (Chroot _ _ h _) = containerProperties h + containerInfo (Chroot _ _ h _) = containerInfo h -chrootSystem :: Chroot -> Maybe System +chrootSystem :: Chroot inner outer -> Maybe System chrootSystem = fromInfoVal . fromInfo . containerInfo -instance Show Chroot where - show c@(Chroot loc _ _) = "Chroot " ++ loc ++ " " ++ show (chrootSystem c) +instance Show (Chroot inner outer) where + show c@(Chroot loc _ _ _) = "Chroot " ++ loc ++ " " ++ show (chrootSystem c) -- | Class of things that can do initial bootstrapping of an operating -- System in a chroot. -class ChrootBootstrapper b where +class ChrootBootstrapper b inner outer where -- | Do initial bootstrapping of an operating system in a chroot. -- If the operating System is not supported, return -- Left error message. - buildchroot :: b -> Maybe System -> FilePath -> Either String (Property Linux) + buildchroot :: b -> Maybe System -> FilePath -> Either String (Property outer) -- | Use this to bootstrap a chroot by extracting a tarball. -- @@ -68,9 +71,8 @@ -- detect automatically. data ChrootTarball = ChrootTarball FilePath -instance ChrootBootstrapper ChrootTarball where - buildchroot (ChrootTarball tb) _ loc = Right $ - tightenTargets $ extractTarball loc tb +instance ChrootBootstrapper ChrootTarball UnixLike UnixLike where + buildchroot (ChrootTarball tb) _ loc = Right $ extractTarball loc tb extractTarball :: FilePath -> FilePath -> Property UnixLike extractTarball target src = check (unpopulated target) $ @@ -88,7 +90,7 @@ -- | Use this to bootstrap a chroot with debootstrap. data Debootstrapped = Debootstrapped Debootstrap.DebootstrapConfig -instance ChrootBootstrapper Debootstrapped where +instance ChrootBootstrapper Debootstrapped DebianLike Linux where buildchroot (Debootstrapped cf) system loc = case system of (Just s@(System (Debian _) _)) -> Right $ debootstrap s (Just s@(System (Buntish _) _)) -> Right $ debootstrap s @@ -107,13 +109,22 @@ -- > & osDebian Unstable "amd64" -- > & Apt.installed ["ghc", "haskell-platform"] -- > & ... -debootstrapped :: Debootstrap.DebootstrapConfig -> FilePath -> Chroot +-- debootstrapped :: Debootstrap.DebootstrapConfig -> FilePath -> Chroot DebianLike +debootstrapped + :: (SingI inner, SingI outer, ChrootBootstrapper Debootstrapped (MetaTypes inner) (MetaTypes outer)) + => Debootstrap.DebootstrapConfig + -> FilePath + -> Chroot (MetaTypes inner) (MetaTypes outer) debootstrapped conf = bootstrapped (Debootstrapped conf) -- | Defines a Chroot at the given location, bootstrapped with the -- specified ChrootBootstrapper. -bootstrapped :: ChrootBootstrapper b => b -> FilePath -> Chroot -bootstrapped bootstrapper location = Chroot location bootstrapper h +bootstrapped + :: (SingI inner, SingI outer, ChrootBootstrapper b (MetaTypes inner) (MetaTypes outer)) + => b + -> FilePath + -> Chroot (MetaTypes inner) (MetaTypes outer) +bootstrapped bootstrapper location = Chroot location bootstrapper h (sing, sing) where h = Host location [] mempty @@ -123,45 +134,79 @@ -- Reverting this property removes the chroot. Anything mounted inside it -- is first unmounted. Note that it does not ensure that any processes -- that might be running inside the chroot are stopped. -provisioned :: Chroot -> RevertableProperty (HasInfo + Linux) Linux +-- provisioned :: SingI outer => Chroot inner outer -> RevertableProperty (HasInfo + MetaTypes outer) Linux +provisioned + :: + ( SingI outer + , SingI metatypes + , Combines (Property (MetaTypes outer)) (Property (MetaTypes outer)) + , (HasInfo + outer) ~ MetaTypes metatypes + , CombinedType (Property (MetaTypes outer)) (Property (MetaTypes outer)) ~ Property outer + , IncludesInfo (MetaTypes metatypes) ~ 'True) + => Chroot inner outer -> RevertableProperty (HasInfo + outer) Linux provisioned c = provisioned' (propagateChrootInfo c) c False provisioned' - :: (Property Linux -> Property (HasInfo + Linux)) - -> Chroot + :: + ( Combines (Property (MetaTypes outer)) (Property (MetaTypes outer)) + , CombinedType (Property (MetaTypes outer)) (Property (MetaTypes outer)) ~ Property outer + , SingI outer + ) + => (Property outer -> Property (HasInfo + outer)) + -> Chroot inner outer -> Bool - -> RevertableProperty (HasInfo + Linux) Linux -provisioned' propigator c@(Chroot loc bootstrapper _) systemdonly = - (propigator $ setup `describe` chrootDesc c "exists") + -> RevertableProperty (HasInfo + outer) Linux +provisioned' propigator c systemdonly = + (propigator $ setup c systemdonly `describe` chrootDesc c "exists") <!> - (teardown `describe` chrootDesc c "removed") - where - setup :: Property Linux - setup = propellChroot c (inChrootProcess (not systemdonly) c) systemdonly - `requires` built - - built = case buildchroot bootstrapper (chrootSystem c) loc of - Right p -> p - Left e -> cantbuild e - - cantbuild e = property (chrootDesc c "built") (error e) - - teardown :: Property Linux - teardown = check (not <$> unpopulated loc) $ - property ("removed " ++ loc) $ - makeChange (removeChroot loc) - -propagateChrootInfo :: Chroot -> Property Linux -> Property (HasInfo + Linux) -propagateChrootInfo c@(Chroot location _ _) p = propagateContainer location c $ - p `addInfoProperty` chrootInfo c + (teardown c `describe` chrootDesc c "removed") -chrootInfo :: Chroot -> Info -chrootInfo (Chroot loc _ h) = mempty `addInfo` +-- chroot removal code is currently linux specific.. +teardown :: Chroot inner outer -> Property Linux +teardown (Chroot loc _ _ _) = check (not <$> unpopulated loc) $ + property ("removed " ++ loc) $ + makeChange (removeChroot loc) + +setup + :: + ( SingI outer + , Combines (Property (MetaTypes outer)) (Property (MetaTypes outer)) + ) + => Chroot inner outer + -> Bool + -> CombinedType (Property (MetaTypes outer)) (Property (MetaTypes outer)) +setup c systemdonly = propellChroot c (inChrootProcess (not systemdonly) c) systemdonly + `requires` built c + +built :: (SingI outer, ChrootBootstrapper b inner outer) => Chroot inner outer -> Property (MetaTypes outer) +built c@(Chroot loc bootstrapper _ _) = + case buildchroot bootstrapper (chrootSystem c) loc of + Right p -> error "FOO" -- p + Left e -> error "FOO" -- cantbuild c e + +cantbuild :: Chroot inner outer -> String -> Property UnixLike +cantbuild c e = property (chrootDesc c "built") (error e) + +propagateChrootInfo + :: + ( SingI metatypes + , (HasInfo + outer) ~ MetaTypes metatypes + , IncludesInfo (MetaTypes metatypes) ~ 'True + ) + => Chroot inner outer + -> Property outer + -> Property (MetaTypes metatypes) +propagateChrootInfo c@(Chroot location _ _ _) p = + propagateContainer location c $ + p `addInfoProperty` chrootInfo c + +chrootInfo :: Chroot inner outer -> Info +chrootInfo (Chroot loc _ h _) = mempty `addInfo` mempty { _chroots = M.singleton loc h } -- | Propellor is run inside the chroot to provision it. -propellChroot :: Chroot -> ([String] -> IO (CreateProcess, IO ())) -> Bool -> Property UnixLike -propellChroot c@(Chroot loc _ _) mkproc systemdonly = property (chrootDesc c "provisioned") $ do +propellChroot :: SingI outer => Chroot inner outer -> ([String] -> IO (CreateProcess, IO ())) -> Bool -> Property (MetaTypes outer) +propellChroot c@(Chroot loc _ _ _) mkproc systemdonly = property (chrootDesc c "provisioned") $ do let d = localdir </> shimdir c let me = localdir </> "propellor" shim <- liftIO $ ifM (doesDirectoryExist d) @@ -199,8 +244,8 @@ liftIO cleanup return r -toChain :: HostName -> Chroot -> Bool -> IO CmdLine -toChain parenthost (Chroot loc _ _) systemdonly = do +toChain :: HostName -> Chroot inner outer -> Bool -> IO CmdLine +toChain parenthost (Chroot loc _ _ _) systemdonly = do onconsole <- isConsole <$> getMessageHandle return $ ChrootChain parenthost loc systemdonly onconsole @@ -224,8 +269,8 @@ putStrLn $ "\n" ++ show r chain _ _ = errorMessage "bad chain command" -inChrootProcess :: Bool -> Chroot -> [String] -> IO (CreateProcess, IO ()) -inChrootProcess keepprocmounted (Chroot loc _ _) cmd = do +inChrootProcess :: Bool -> Chroot inner outer -> [String] -> IO (CreateProcess, IO ()) +inChrootProcess keepprocmounted (Chroot loc _ _ _) cmd = do mountproc return (proc "chroot" (loc:cmd), cleanup) where @@ -244,26 +289,24 @@ provisioningLock :: FilePath -> FilePath provisioningLock containerloc = "chroot" </> mungeloc containerloc ++ ".lock" -shimdir :: Chroot -> FilePath -shimdir (Chroot loc _ _) = "chroot" </> mungeloc loc ++ ".shim" +shimdir :: Chroot inner outer -> FilePath +shimdir (Chroot loc _ _ _) = "chroot" </> mungeloc loc ++ ".shim" mungeloc :: FilePath -> String mungeloc = replace "/" "_" -chrootDesc :: Chroot -> String -> String -chrootDesc (Chroot loc _ _) desc = "chroot " ++ loc ++ " " ++ desc +chrootDesc :: Chroot inner outer -> String -> String +chrootDesc (Chroot loc _ _ _) desc = "chroot " ++ loc ++ " " ++ desc
2016-03-26ported dockerJoey Hess
Also, implemented modifyHostProps to add properties to an existing host. Using it bypasses some type safety. Its use in docker is safe though. But, in Conductor, the use of it was not really safe, because it was used with a DebianLike property. Fixed that by making Ssh.installed target all unix's, although it will fail on non-DebianLike ones.
2015-12-06allow using `check` on a UncheckedProperty, which yields a PropertyJoey Hess
2015-12-05UncheckedProperty for cmdProperty et alJoey Hess
* Properties that run an arbitrary command, such as cmdProperty and scriptProperty are converted to use UncheckedProperty, since they cannot tell on their own if the command truely made a change or not. (API Change) Transition guide: - When GHC complains about an UncheckedProperty, add: `assume` MadeChange - Since these properties used to always return MadeChange, that change is always safe to make. - Or, if you know that the command should modifiy a file, use: `changesFile` filename * A few properties have had their Result improved, for example Apt.buldDep and Apt.autoRemove now check if a change was made or not.
2015-11-24haddock improvementsJoey Hess
2015-11-17DiskImage creation automatically uses Chroot.noServices.Joey Hess
2015-10-27Explicit Info/NoInfo for RevertableProperty (API change)Joey Hess
RevertableProperty used to be assumed to contain info, but this is now made explicit, with RevertableProperty HasInfo or RevertableProperty NoInfo. Transition guide: - If you define a RevertableProperty, expect some type check failures like: "Expecting one more argument to ‘RevertableProperty’". - Change it to "RevertableProperty NoInfo" - The compiler will then tell you if it needs "HasInfo" instead. - If you have code that uses the RevertableProperty constructor that fails to type check, use the more powerful <!> operator
2015-10-23allow specifying filesystem mount optionsJoey Hess
2015-10-23refactorJoey Hess