OsmAnd has full native GPX support, and it's genuinely simple once your list is converted. Paste your Google Maps shared list link into the converter above and export as GPX.
Fastest route, the share sheet: open the GPX file from your phone's Files app, tap Share (or long-press → Open in), and choose OsmAnd. It opens automatically and drops the file into the Import folder under My Places → Tracks.
Alternative, import from inside the app: open the My Places menu, go to Tracks, select Import, then browse to your GPX file.
Once it's in, your points are available offline, no signal needed once the file's loaded and your map area is downloaded.
Prefer another free offline map app? The Organic Maps page covers importing the same GPX file there.