Related issues:
#15717
#22429
#29364
#33672

Some more context: I'm not sure that we're getting a lot of mileage out of the bundled pub cache. This will only help developers with an offline mode until they add their first plugin, then they need to pull dependencies for Gradle/Cocoapods too

I would like to retain the property that the SDK bundles contain most or all of what needs to be downloaded from Flutter/Dart to get started. As noted elsewhere, there was a measurable uptick in retention concomitant with that change. This implies that the first time the tool runs from an SDK bundle, it will need to do some surgery to migrate to $HOME/.pub-cache. That process will need good test coverage, which could be the more difficult part of the change.

I would like to retain the property that the SDK bundles contain most or all of what needs to be downloaded from Flutter/Dart to get started.

The safest option is for you to create a tarball containing a pre-populated $PUB_CACHE and ONLY copy it into place if there is no $PUB_CACHE already. You can find the default path for $PUB_CACHE using:

https://github.com/dart-lang/pub/blob/3606265962da4248d34d352aa3d170aae4496a90/lib/src/system_cache.dart#L34-L53

Beware that there is special tricks on place on Windows.

You can also extract specific package tarballs into $PUB_CACHE/hosted/pub.dartlang.org/<package>-<version>/. But I can't promise that the layout of the $PUB_CACHE won't change in the future -- so if you do this I would recommend writing tests that covers this.

One case where this hydration strategy worsens the situation is when a user had previously tried Flutter or Dart and has a pub cache as a result. I don't have an idea of how common this would be, but we could avoid making the situation worse if there were a way to populate an existing cache.

@jonasfj though I don't think the cache structure of pub is necessarily a public API, we do ship the flutter tool in lockstep with pub. Would it be reasonable for the tool to make some assumptions here and still preserve safety?

but we could avoid making the situation worse if there were a way to populate an existing cache.

I would suggest not populating the cache if there is already one present. In this case it's likely many of the packages are already there :D
If we really do want to do this, it wouldn't be harmful, so long as we just extract tarballs into $PUB_CACHE/hosted/pub.dartlang.org/<package>-<version>/ (assuming said directory doesn't exist.

Would it be reasonable for the tool to make some assumptions here and still preserve safety?

We can probably promise that if the PUB_CACHE directory was created by a previous version of pub it will work with a future version of pub (or at-least not cause issues for a future version of pub).

Hence, if you only create the cache when no $PUB_CACHE folder exists, and follow the pattern that PUB_CACHE currently has, then this should always work. Even if a future version of pub decides to change the cache-layout, such newer version of pub will just think that the PUB_CACHE was created by a previous version of pub.

This is likely contributing to Dart analyzer performance issues for mono-repo projects where pub get was run differently for different packages in the mono_repo.

Can you elaborate, @jacob314 ?

TLDR: The analyzer relies on absolute file system paths to determine whether two versions of a package are the same. If your mono-repo has two pub cache directories, 2X the memory and CPU is used for resolving packages that are used from both pub cache directories.

If you have a mono-repo with some Dart packages (for example: devtools_shared) and some Flutter packages (for examples: devtools_app) and you happen to run
dart pub get from your Dart package and flutter pub get from your Flutter package then you will have the Flutter specific pub cache directory for one package and the Dart one for your other package. If you next open the parent directory of devtools_app and devtools_shared in an IDE, the Analyzer will be unable to tell that the shared dependencies of the two packages are actually shared because they are in different directories. This doubles the memory usage from the analyzer loading and resolving the shared packages and doubles the time spent resolving the shared packages.

After having a discussion with @jacob314. @christopherfujino and I believe the best approach to handle this case is to copy your pub cache located at $FLUTTER_ROOT/.pub-cache over to $HOME/.pub-cache only when $HOME/.pub-cache is not found and then proceed to delete the $FLUTTER_ROOT/.pub-cache directory. The question still remains in what part of the process should be more adequate to add the change. Current idea is when doing a flutter pub get but only the first time a user downloads Flutter.

Yes, I think something like that makes sense; WDYT @jonasfj ?

WDYT @jonasfj ?

Love it!

Not having two PUB_CACHE locations will make the world simpler.

breadcrumbs: before implementing this, further discussion is needed to find a good strategy for testing the migration