always return complete ways on getMapDataWithGeometry

fixes #4980

`MapDataCache` can now fetch nodes if for some reason it's neither in `SpatialCache` nor in `nodeCache`. This can be because of trim (simply clears `nodeCache`), but also for other reasons, e.g. `SpatialCache` removing tiles unexpectedly (see #4985).

Performance:
* `getMatDataWithGeometry` is slower
  * for large bboxes (usually for overlays) 20-30% of total time is spend on finding / adding nodes needed for complete ways
  * for small bboxes (highlighted elements), this is more, around 30-90%. This may seem much, but even on a S4 mini and a debug APK data is returned within 5-20 ms
* `update` is typically around 10% slower due to properly handling `nodeCache` (25% observed, this was by far the worst case)
* fetching nodes happens only after trim and after encountering the `tilesRect.asBoundingBox(zoom).enclosingTilesRect(zoom)` issue
  * fetched nodes are cached in `nodeCache`, so even after trim this fetch does not happen often
  * for small bboxes, fetching nodes (if necessary) takes typically 50% of the total time of `getMatDataWithGeometry`
  * for large bboxes (overlays), it's more like 15-20% (up to 60% observed)

IMO the performance impact is acceptable, and likely any improvements will not be worth the effort.

Some work is still to be done: No new tests are added, and some old tests fail now.

`MapDataController::getMapDataWithGeometry` returns incomplete Ways
I think as part of @Helium314 's extensive caching PR or even a bit earlier than than, we made bounding box queries to local map data mimic the behavior of the OpenStreetMap API:

1. get all nodes in bounding box
2. get all ways that use any of these nodes
3. get all relations that use any of these nodes or any of these ways

However, we have overlooked something: The OpenStreetMap API never returns incomplete ways. In other words, if a way is included in the result set, it always also includes all nodes of the way, even if some of these nodes are not in the given bounding box. Currently, `MapDataController::getMapDataWithGeometry` returns all ways that have nodes within the given bounding box, but omits nodes of these ways that are not within the given bounding box.

This inconsistency has not been deliberate, but an oversight (at least from my part). I am pretty sure this is the source of #4925, maybe others. It definitely affects #4976.

A fix would look like this in the `ElementDao` (plus tests)...

```diff
--- a/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDao.kt
+++ b/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDao.kt
@@ -45,12 +45,19 @@ class ElementDao(

     fun getAll(bbox: BoundingBox): List<Element> {
         val nodes = nodeDao.getAll(bbox)
-        val nodeIds = nodes.map { it.id }
+        val nodeIds = nodes.map { it.id }.toSet()
         val ways = wayDao.getAllForNodes(nodeIds)
         val wayIds = ways.map { it.id }
-        val relations = relationDao.getAllForElements(nodeIds = nodeIds, wayIds = wayIds)
-        val result = ArrayList<Element>(nodes.size + ways.size + relations.size)
+        val additionalWayNodeIds = ways
+            .asSequence()
+            .flatMap { it.nodeIds }
+            .filter { it !in nodeIds }
+            .toList()
+        val additionalNodes = nodeDao.getAll(additionalWayNodeIds)
+        val relations = relationDao.getAllForElements(nodeIds = additionalWayNodeIds + nodeIds, wayIds = wayIds)
+        val result = ArrayList<Element>(nodes.size + additionalNodes.size + ways.size + relations.size)
         result.addAll(nodes)
+        result.addAll(additionalNodes)
         result.addAll(ways)
         result.addAll(relations)
         return result
```

but I think the `MapDataCache` also would need to be changed somewhat. I looked a bit into that, and it'd be more than a few lines of code. @Helium314, what do you think? Do you think my assessment is correct? And if yes, would you be willing to take care of changing the `MapDataCache` to this effect?

Adjusting the cache would most likely need some more work... sure, I would do it.

Currently nodes are only stored in `SpatialCache`, but this won't be feasible if we also want to cache nodes that basically might be anywhere.
My idea is adding some cache (like `wayCache` and `relationCache` for those extra nodes, but it will be a bit tricky making this work nicely with adding/removing/trimming tiles from `SpatialCache`.

I think the `MapDataCache` should get the ability to fetch nodes from DB. This would remove the need to solve every edge case, as sometimes it might be necessary to loop over every node of every cached way (and this might be slower than fetching a few nodes from db).

[Edit: I overlooked the `wayIdsByNodeIdCache` that should make things less bad, but still I think simply re-fetching "outside nodes" from DB when necessary would likely simplify things]
Regarding `nodeCache`, I was suspecting something like this when reading the code. I don't follow though why the `MapDataCache` should have (direct??) access to the DB... indirectly, sure, this is how it currently works, isn't it.

If you are saying that fetching nodes outside of the bbox without caching, ... well, this basically would need to be done for every single bbox no? There will always be one `way` or another that is not completely contained within that bbox.
> If you are saying that fetching nodes outside of the bbox without caching, ... well, this basically would need to be done for every single bbox no?

In the initial bbox fetch for those ways, the nodes are included and thus will be in the `nodeCache`.
The DB access is meant for cases where some of the nodes that were in `SpatialCache` are removed.

Though maybe if `SpatialCache.trim` would return removed nodes, it should be possible to find necessary ones and add them to the `nodeCache`.
But when trimming because of memory pressure, responding with a (temporary and likely small) increase of memory use might be bad.
Biggest potential issue when `MapDataCache` is not allowed to fetch nodes:
`SpatialCache` may remove tiles without explicitly calling `trim`. This can be when on `replaceAllInBBox` and `get(bbox)` call `trim`, or when `replaceAllInBBox` removes incomplete tiles.
But `MapDataCache` would need to know if anything is removed from `SpatialCache`, and put it to `nodeCache` if it's part of a cached way.

So even returning removed nodes on `trim` and putting some of them to `nodeCache` likely won't guarantee that all nodes of cached ways remain cached.

See https://github.com/streetcomplete/StreetComplete/compare/master...Helium314:SCEE:complete_ways for more comments.
Disclaimer, the cache is your brainchild, which is why
1. my understanding of it without deeply putting my head into it is limited and
2. I guess fetching the nodes from DB will result in somewhat of a performance impact because it would have to be done on virtually every bbox query and I know that performance optimization is very important to you. So if you say that caching nodes outside their bbox is not feasible, I trust your assessment.

And, I guess, optimizations could always be done later, after the fix. If we have a working version first, we could also have tests for that first, so when finally coming to the performance optimizations, we can be more sure that everything is working as expected because it is covered by tests.
In any case, maybe this is a case where test-driven development would be useful. There should be tests anyway, might as well start with them.

>  private val nodeCache = HashMap<Long, Node>() // what capacity? some estimate would be useful...

`initialCapacity`

I've looked at the other `todos` but I can't be of any help without wrapping my head completely around the caching stuff again.

So I'll start with fetching nodes from DB and simply clearing nodeCache on trim, and do some performance logging.
Current version of moving some nodes from spatialCache to nodeCache on trim is ridiculously slow.
After some testing I think fetching nodes from DB is fine. It's necessary only after trimming, which does not occur frequently, and those nodes are put to `nodeCache` anyway, so no need to fetch them again until the next trim.
Even if a fetch from node DB is necessary, it usually takes around 50% of the time spent in `getMapDataWithGeometry`. So no huge slowdown.
I stumbled upon a weird issue that might be a bug in the cache. Looks like it's triggered by `tilesRect.asBoundingBox(zoom).enclosingTilesRect(zoom)`  sometimes not being equal to the intial `rect`.

e.g. for `TilesRect(71512, 45464, 71513, 45465)` it's equal at zoom 18, but not at zoom 17.

This leads to incomplete tiles in [`SpatialCache`](https://github.com/streetcomplete/StreetComplete/blob/14519b7e978bfa549e935c882b753a67e7f096ac/app/src/main/java/de/westnordost/streetcomplete/util/SpatialCache.kt#L88-L92).

The issue with the cache seems to be that these removed incomplete tiles are added to SpatialCache again as empty tiles when [fetching](https://github.com/streetcomplete/StreetComplete/blob/14519b7e978bfa549e935c882b753a67e7f096ac/app/src/main/java/de/westnordost/streetcomplete/util/SpatialCache.kt#L117-L120) newly added nodes form `MapDatCache`. On the next fetch, `MapDataCache` only sees that these tiles exist, but actually they are empty (and should _not_ be in `SpatialCache`).
Oh wow, that's cool! I mean, that you maybe found some bug. Bugs in the cache can lead to all sorts of esoteric problems. I suggest you write a test for this and fix it in a separate branch though, it has nothing to do with this here, right?
> it has nothing to do with this here, right?

No, only that I noticed it as part of this work.
Fix for cache is already done, test not.

But the `tilesRect.asBoundingBox(zoom).enclosingTilesRect(zoom)` behavior is a bug too, right? At very least it should be consistent.
Here I have no idea how to fix it... I only remember there were similar issues with bbox -> tilesRect -> bbox which you fixed.
Hmm, not sure about that. Maybe we conversed about this already.

Maybe one of the two is inclusive on one edge and the other is exclusive on one edge. There is some lengthy comment in `TilesRect.kt`
But I think you are right... TilesRect.asBoundingBox should return a bounding box that is floating-point-imprecision smaller than the tilesrect. So if you call BoundingBox.enclosingTilesRect, it should return again the same TilesRect.

./gradlew :app:testDebugUnitTest --tests "de.westnordost.streetcomplete.data.osm.mapdata.ElementDaoTest" --tests "de.westnordost.streetcomplete.data.osm.mapdata.MapDataCacheTest"

diff --git a/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDao.kt b/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDao.kt
index 3fdf0421c08..4e2efdc7399 100644
--- a/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDao.kt
+++ b/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDao.kt
@@ -45,12 +45,19 @@ class ElementDao(

     fun getAll(bbox: BoundingBox): List<Element> {
         val nodes = nodeDao.getAll(bbox)
-        val nodeIds = nodes.map { it.id }
+        val nodeIds = nodes.map { it.id }.toSet()
         val ways = wayDao.getAllForNodes(nodeIds)
         val wayIds = ways.map { it.id }
-        val relations = relationDao.getAllForElements(nodeIds = nodeIds, wayIds = wayIds)
-        val result = ArrayList<Element>(nodes.size + ways.size + relations.size)
+        val additionalWayNodeIds = ways
+            .asSequence()
+            .flatMap { it.nodeIds }
+            .filter { it !in nodeIds }
+            .toList()
+        val additionalNodes = nodeDao.getAll(additionalWayNodeIds)
+        val relations = relationDao.getAllForElements(nodeIds = additionalWayNodeIds + nodeIds, wayIds = wayIds)
+        val result = ArrayList<Element>(nodes.size + additionalNodes.size + ways.size + relations.size)
         result.addAll(nodes)
+        result.addAll(additionalNodes)
         result.addAll(ways)
         result.addAll(relations)
         return result
diff --git a/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCache.kt b/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCache.kt
index c4c79d7f0c1..bf913878742 100644
--- a/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCache.kt
+++ b/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCache.kt
@@ -7,6 +7,7 @@ import de.westnordost.streetcomplete.data.osm.geometry.ElementGeometryEntry
 import de.westnordost.streetcomplete.data.osm.geometry.ElementPointGeometry
 import de.westnordost.streetcomplete.util.SpatialCache
 import de.westnordost.streetcomplete.util.math.contains
+import de.westnordost.streetcomplete.util.math.isCompletelyInside

 /**
  * Cache for MapDataController that uses SpatialCache for nodes (i.e. geometry) and hash maps
@@ -22,6 +23,7 @@ class MapDataCache(
     val maxTiles: Int,
     initialCapacity: Int,
     private val fetchMapData: (BoundingBox) -> Pair<Collection<Element>, Collection<ElementGeometryEntry>>, // used if the tile is not contained
+    private val fetchNodes: (Collection<Long>) -> Collection<Node>,
 ) {
     private val spatialCache = SpatialCache(
         tileZoom,
@@ -34,6 +36,7 @@ class MapDataCache(
     //  approximately 80% of all elements were found to be nodes
     //  approximately every second node is part of a way
     //  more than 90% of elements are not part of a relation
+    private val nodesOutsideSpatialCache = HashMap<Long, Node>(initialCapacity)
     private val wayCache = HashMap<Long, Way>(initialCapacity / 6)
     private val relationCache = HashMap<Long, Relation>(initialCapacity / 10)
     private val wayGeometryCache = HashMap<Long, ElementGeometry>(initialCapacity / 6)
@@ -64,12 +67,15 @@ class MapDataCache(
             spatialCache.update(deleted = deletedNodeIds)
             spatialCache.replaceAllInBBox(updatedNodes, bbox)
         }
+        // remove all cached nodes that are now in spatialCache
+        nodesOutsideSpatialCache.keys.removeAll { spatialCache.get(it) != null }

         // delete nodes, ways and relations
         for (key in deletedKeys) {
             when (key.type) {
                 ElementType.NODE -> {
                     wayIdsByNodeIdCache.remove(key.id)
+                    nodesOutsideSpatialCache.remove(key.id)
                 }
                 ElementType.WAY -> {
                     val deletedWayNodeIds = wayCache.remove(key.id)?.nodeIds.orEmpty()
@@ -97,6 +103,9 @@ class MapDataCache(
             }
         }

+        // add nodes that are not in spatialCache to nodeCache
+        updatedNodes.forEach { if (spatialCache.get(it.id) == null) nodesOutsideSpatialCache[it.id] = it }
+
         // update ways
         val updatedWays = updatedElements.filterIsInstance<Way>()
         for (way in updatedWays) {
@@ -178,8 +187,8 @@ class MapDataCache(
         id: Long,
         fetch: (ElementType, Long) -> Element?
     ): Element? = synchronized(this) {
-        return when (type) {
-            ElementType.NODE -> spatialCache.get(id) ?: fetch(type, id)
+        when (type) {
+            ElementType.NODE -> spatialCache.get(id) ?: nodesOutsideSpatialCache.getOrPutIfNotNull(id) { fetch(type, id) as? Node }
             ElementType.WAY -> wayCache.getOrPutIfNotNull(id) { fetch(type, id) as? Way }
             ElementType.RELATION -> relationCache.getOrPutIfNotNull(id) { fetch(type, id) as? Relation }
         }
@@ -195,7 +204,7 @@ class MapDataCache(
         fetch: (ElementType, Long) -> ElementGeometry?
     ): ElementGeometry? = synchronized(this) {
         return when (type) {
-            ElementType.NODE -> spatialCache.get(id)?.let { ElementPointGeometry(it.position) } ?: fetch(type, id)
+            ElementType.NODE -> getCachedNode(id)?.let { ElementPointGeometry(it.position) } ?: fetch(type, id)
             ElementType.WAY -> wayGeometryCache.getOrPutIfNotNull(id) { fetch(type, id) }
             ElementType.RELATION -> relationGeometryCache.getOrPutIfNotNull(id) { fetch(type, id) }
         }
@@ -213,7 +222,7 @@ class MapDataCache(
     ): List<Element> = synchronized(this) {
         val cachedElements = keys.mapNotNull { key ->
             when (key.type) {
-                ElementType.NODE -> spatialCache.get(key.id)
+                ElementType.NODE -> getCachedNode(key.id)
                 ElementType.WAY -> wayCache[key.id]
                 ElementType.RELATION -> relationCache[key.id]
             }
@@ -228,9 +237,9 @@ class MapDataCache(
         val fetchedElements = fetch(keysToFetch)
         for (element in fetchedElements) {
             when (element.type) {
+                ElementType.NODE -> nodesOutsideSpatialCache[element.id] = element as Node
                 ElementType.WAY -> wayCache[element.id] = element as Way
                 ElementType.RELATION -> relationCache[element.id] = element as Relation
-                else -> Unit
             }
         }
         return cachedElements + fetchedElements
@@ -239,13 +248,14 @@ class MapDataCache(
     /** Gets the nodes with the given [ids] from cache. If any of the nodes are not cached, [fetch]
      *  is called for the missing nodes. */
     fun getNodes(ids: Collection<Long>, fetch: (Collection<Long>) -> List<Node>): List<Node> = synchronized(this) {
-        val cachedNodes = spatialCache.getAll(ids)
+        val cachedNodes = ids.mapNotNull { getCachedNode(it) }
         if (ids.size == cachedNodes.size) return cachedNodes

         // not all in cache: must fetch the rest from db
         val cachedNodeIds = cachedNodes.map { it.id }.toSet()
         val missingNodeIds = ids.filterNot { it in cachedNodeIds }
         val fetchedNodes = fetch(missingNodeIds)
+        fetchedNodes.forEach { nodesOutsideSpatialCache[it.id] = it }
         return cachedNodes + fetchedNodes
     }

@@ -279,7 +289,7 @@ class MapDataCache(
         // that geometries and not elements are returned and thus different caches are accessed
         val cachedEntries = keys.mapNotNull { key ->
             when (key.type) {
-                ElementType.NODE -> spatialCache.get(key.id)?.let { ElementPointGeometry(it.position) }
+                ElementType.NODE -> getCachedNode(key.id)?.let { ElementPointGeometry(it.position) }
                 ElementType.WAY -> wayGeometryCache[key.id]
                 ElementType.RELATION -> relationGeometryCache[key.id]
             }?.let { ElementGeometryEntry(key.type, key.id, it) }
@@ -404,10 +414,33 @@ class MapDataCache(
             relationIdsByElementKeyCache[node.key]?.let { relationIds.addAll(it) }
             result.put(node, ElementPointGeometry(node.position))
         }
+
+        val nodesToFetch = hashSetOf<Long>()
         for (wayId in wayIds) {
-            result.put(wayCache[wayId]!!, wayGeometryCache[wayId])
-            relationIdsByElementKeyCache[ElementKey(ElementType.WAY, wayId)]?.let { relationIds.addAll(it) }
+            val way = wayCache[wayId]!!
+            val wayGeometry = wayGeometryCache[wayId]
+            result.put(way, wayGeometry)
+            relationIdsByElementKeyCache[way.key]?.let { relationIds.addAll(it) }
+
+            // find all nodes that are part of the way, but not in result
+            if (wayGeometry?.getBounds()?.isCompletelyInside(bbox) == true) continue // no need to check
+            for (nodeId in way.nodeIds) {
+                if (result.getNode(nodeId) != null) continue
+                val cachedNode = getCachedNode(nodeId)
+                if (cachedNode != null) {
+                    result.put(cachedNode, ElementPointGeometry(cachedNode.position))
+                    continue
+                }
+                nodesToFetch.add(nodeId)
+            }
+        }
+        if (nodesToFetch.isNotEmpty()) {
+            fetchNodes(nodesToFetch).forEach {
+                nodesOutsideSpatialCache[it.id] = it
+                result.put(it, ElementPointGeometry(it.position))
+            }
         }
+
         for (relationId in relationIds) {
             result.put(relationCache[relationId]!!, relationGeometryCache[relationId])
             // don't add relations of relations, because elementDao.getAll(bbox) also isn't doing that
@@ -424,6 +457,7 @@ class MapDataCache(
     /** Clears the cache */
     fun clear() { synchronized(this) {
         spatialCache.clear()
+        nodesOutsideSpatialCache.clear()
         wayCache.clear()
         relationCache.clear()
         wayGeometryCache.clear()
@@ -437,6 +471,7 @@ class MapDataCache(
      */
     fun trim(tiles: Int) { synchronized(this) {
         spatialCache.trim(tiles)
+        nodesOutsideSpatialCache.clear() // simply clear nodeCache, as transferring some nodes from spatialCache is slow

         // ways and relations with at least one element in cache should not be removed
         val (wayIds, relationIds) = determineWayAndRelationIdsWithElementsInSpatialCache()
@@ -501,4 +536,6 @@ class MapDataCache(

     private fun Node.toElementGeometryEntry() =
         ElementGeometryEntry(type, id, ElementPointGeometry(position))
+
+    private fun getCachedNode(id: Long): Node? = spatialCache.get(id) ?: nodesOutsideSpatialCache[id]
 }
diff --git a/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataController.kt b/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataController.kt
index 8375fbbfe78..d3e4915c51c 100644
--- a/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataController.kt
+++ b/app/src/main/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataController.kt
@@ -41,14 +41,16 @@ class MapDataController internal constructor(
     private val cache = MapDataCache(
         SPATIAL_CACHE_TILE_ZOOM,
         SPATIAL_CACHE_TILES,
-        SPATIAL_CACHE_INITIAL_CAPACITY
-    ) { bbox ->
-        val elements = elementDB.getAll(bbox)
-        val elementGeometries = geometryDB.getAllEntries(
-            elements.mapNotNull { if (it !is Node) it.key else null }
-        )
-        elements to elementGeometries
-    }
+        SPATIAL_CACHE_INITIAL_CAPACITY,
+        { bbox ->
+            val elements = elementDB.getAll(bbox)
+            val elementGeometries = geometryDB.getAllEntries(
+                elements.mapNotNull { if (it !is Node) it.key else null }
+            )
+            elements to elementGeometries
+        },
+        { nodeDB.getAll(it) },
+    )

     /** update element data with [mapData] in the given [bbox] (fresh data from the OSM API has been
      *  downloaded) */

diff --git a/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDaoTest.kt b/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDaoTest.kt
index 4b4ca12c546..fa017a0db85 100644
--- a/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDaoTest.kt
+++ b/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/ElementDaoTest.kt
@@ -139,7 +139,7 @@ class ElementDaoTest {
         val relations = listOf(rel(1))

         on(nodeDao.getAll(bbox)).thenReturn(nodes)
-        on(wayDao.getAllForNodes(eq(nodeIds))).thenReturn(ways)
+        on(wayDao.getAllForNodes(eq(nodeIds.toSet()))).thenReturn(ways)
         on(relationDao.getAllForElements(
             nodeIds = eq(nodeIds),
             wayIds = eq(wayIds),
@@ -151,6 +151,30 @@ class ElementDaoTest {
         )
     }

+    @Test fun `getAllElementsByBbox includes nodes that are not in bbox, but part of ways contained in bbox`() {
+        val bbox = BoundingBox(0.0, 0.0, 1.0, 1.0)
+        val bboxNodes = listOf(node(1), node(2), node(3))
+        val bboxNodeIds = bboxNodes.map { it.id }
+        val outsideBboxNodes = listOf(node(4), node(5))
+        val outsideBboxNodeIds = outsideBboxNodes.map { it.id }
+        val ways = listOf(way(1), way(2, nodes = listOf(3L, 4L, 5L)))
+        val wayIds = ways.map { it.id }
+        val relations = listOf(rel(1))
+
+        on(nodeDao.getAll(bbox)).thenReturn(bboxNodes)
+        on(nodeDao.getAll(outsideBboxNodeIds)).thenReturn(outsideBboxNodes)
+        on(wayDao.getAllForNodes(eq(bboxNodeIds.toSet()))).thenReturn(ways)
+        on(relationDao.getAllForElements(
+            nodeIds = eq(outsideBboxNodeIds + bboxNodeIds),
+            wayIds = eq(wayIds),
+            relationIds = eq(emptyList())
+        )).thenReturn(relations)
+        assertEquals(
+            bboxNodes + outsideBboxNodes + ways + relations,
+            dao.getAll(bbox)
+        )
+    }
+
     @Test fun getAllElementKeysByBbox() {
         val bbox = BoundingBox(0.0, 0.0, 1.0, 1.0)
         val nodeIds = listOf<Long>(1, 2, 3)
diff --git a/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCacheTest.kt b/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCacheTest.kt
index e0c41f9d2b6..b15a7e933e5 100644
--- a/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCacheTest.kt
+++ b/app/src/test/java/de/westnordost/streetcomplete/data/osm/mapdata/MapDataCacheTest.kt
@@ -12,13 +12,14 @@ import de.westnordost.streetcomplete.util.math.isCompletelyInside
 import org.junit.Assert.*
 import org.junit.Test
 import org.mockito.Mockito.verify
+import org.mockito.Mockito.verifyNoInteractions
 import org.mockito.Mockito.verifyNoMoreInteractions

 internal class MapDataCacheTest {

     @Test fun `update puts way`() {
         val way = way(1)
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(way))
         val elementDB: ElementDao = mock()
         on(elementDB.get(ElementType.WAY, 1L)).thenThrow(IllegalStateException())
@@ -27,7 +28,7 @@ internal class MapDataCacheTest {

     @Test fun `update puts relation`() {
         val relation = rel(1)
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(relation))
         val elementDB: ElementDao = mock()
         on(elementDB.get(ElementType.RELATION, 1L)).thenThrow(IllegalStateException())
@@ -38,7 +39,7 @@ internal class MapDataCacheTest {
         val p = p(0.0, 0.0)
         val geo = ElementPolylinesGeometry(listOf(listOf(p)), p)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedGeometries = listOf(ElementGeometryEntry(ElementType.WAY, 1, geo)))
         val geometryDB: ElementGeometryDao = mock()
         on(geometryDB.get(ElementType.WAY, 1L)).thenThrow(IllegalStateException())
@@ -49,31 +50,31 @@ internal class MapDataCacheTest {
         val p = p(0.0, 0.0)
         val geo = ElementPolygonsGeometry(listOf(listOf(p)), p)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedGeometries = listOf(ElementGeometryEntry(ElementType.RELATION, 1, geo)))
         val geometryDB: ElementGeometryDao = mock()
         on(geometryDB.get(ElementType.RELATION, 1L)).thenThrow(IllegalStateException())
         assertEquals(geo, cache.getGeometry(ElementType.RELATION, 1L) { type, id -> geometryDB.get(type, id) })
     }

-    @Test fun `getElement fetches node if not in spatialCache`() {
+    @Test fun `getElement also caches node if not in spatialCache`() {
         val node = node(1)
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val elementDB: ElementDao = mock()
         on(elementDB.get(ElementType.NODE, 1L)).thenReturn(node).thenReturn(null)
         assertEquals(node, cache.getElement(ElementType.NODE, 1L) { type, id -> elementDB.get(type, id) })
         verify(elementDB).get(ElementType.NODE, 1L)

-        // getting a second time fetches again
+        // getting a second time does not fetches again
         val elementDB2: ElementDao = mock()
         on(elementDB2.get(ElementType.NODE, 1L)).thenReturn(node).thenReturn(null)
         assertEquals(node, cache.getElement(ElementType.NODE, 1L) { type, id -> elementDB2.get(type, id) })
-        verify(elementDB2).get(ElementType.NODE, 1L)
+        verifyNoInteractions(elementDB2)
     }

     @Test fun `getElement fetches and caches way`() {
         val way = way(2L)
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val elementDB: ElementDao = mock()
         on(elementDB.get(ElementType.WAY, 2L)).thenReturn(way).thenThrow(IllegalStateException())
         // get way 2 and verify the fetch function is called, but only once
@@ -86,7 +87,7 @@ internal class MapDataCacheTest {

     @Test fun `getElement fetches and caches relation`() {
         val rel = rel(1L)
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val elementDB: ElementDao = mock()
         on(elementDB.get(ElementType.RELATION, 1L)).thenReturn(rel).thenThrow(IllegalStateException())
         // get rel 1 and verify the fetch function is called, but only once
@@ -101,7 +102,7 @@ internal class MapDataCacheTest {
         val p = p(0.0, 0.0)
         val geo = ElementPointGeometry(p)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val geometryDB: ElementGeometryDao = mock()
         on(geometryDB.get(ElementType.NODE, 2L)).thenReturn(geo).thenThrow(IllegalStateException())
         // get node 2 and verify the fetch function is called, but only once
@@ -119,7 +120,7 @@ internal class MapDataCacheTest {
         val p = p(0.0, 0.0)
         val geo = ElementPolylinesGeometry(listOf(listOf(p)), p)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val geometryDB: ElementGeometryDao = mock()
         on(geometryDB.get(ElementType.WAY, 2L)).thenReturn(geo).thenThrow(IllegalStateException())
         // get geo and verify the fetch function is called, but only once
@@ -134,7 +135,7 @@ internal class MapDataCacheTest {
         val p = p(0.0, 0.0)
         val geo = ElementPolygonsGeometry(listOf(listOf(p)), p)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val geometryDB: ElementGeometryDao = mock()
         on(geometryDB.get(ElementType.RELATION, 2L)).thenReturn(geo).thenThrow(IllegalStateException())
         // get way 2 and verify the fetch function is called, but only once
@@ -153,7 +154,7 @@ internal class MapDataCacheTest {
         val nodesRect = listOf(node1.position, node2.position, node3.position).enclosingBoundingBox().enclosingTilesRect(16)
         assertTrue(nodesRect.size <= 4) // fits in cache

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = nodes, bbox = nodesRect.asBoundingBox(16))

         assertTrue(cache.getNodes(nodes.map { it.id }) { throw IllegalStateException() }.containsExactlyInAnyOrder(nodes))
@@ -163,34 +164,52 @@ internal class MapDataCacheTest {
         val node1 = node(1, LatLon(0.0, 0.0))
         val node2 = node(2, LatLon(0.0001, 0.0001))
         val node3 = node(3, LatLon(0.0002, 0.0002))
-        val outsideNode = node(4, LatLon(1.0, 1.0))
+        val outsideNode = node(4, LatLon(1.0, 1.0)) // will be put to nodeCache
+        val uncachedNode = node(5, LatLon(1.0, 1.0)) // not in update, so needs to be fetched from DB
         val nodes = listOf(node1, node2, node3, outsideNode)
+        val allNodes = nodes + uncachedNode
         val nodesRect = listOf(node1.position, node2.position, node3.position).enclosingBoundingBox().enclosingTilesRect(16)
         assertTrue(nodesRect.size <= 4) // fits in cache

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = nodes, bbox = nodesRect.asBoundingBox(16))

         val nodeDB: NodeDao = mock()
-        on(nodeDB.getAll(listOf(outsideNode.id))).thenReturn(listOf(outsideNode))
+        on(nodeDB.getAll(listOf(uncachedNode.id))).thenReturn(listOf(uncachedNode))
+        assertTrue(cache.getNodes(allNodes.map { it.id }) { nodeDB.getAll(it) }.containsExactlyInAnyOrder(allNodes))
+        verify(nodeDB).getAll(listOf(uncachedNode.id))
+    }
+
+    @Test fun `getNodes does not fetch cached nodes`() {
+        val node1 = node(1, LatLon(0.0, 0.0))
+        val node2 = node(2, LatLon(0.0001, 0.0001))
+        val node3 = node(3, LatLon(0.0002, 0.0002))
+        val node4 = node(4, LatLon(1.0, 1.0))
+        val nodes = listOf(node1, node2, node3, node4)
+
+        val cache = getEmptyMapDataCache()
+        cache.update(updatedElements = nodes)
+
+        val nodeDB: NodeDao = mock()
         assertTrue(cache.getNodes(nodes.map { it.id }) { nodeDB.getAll(it) }.containsExactlyInAnyOrder(nodes))
-        verify(nodeDB).getAll(listOf(outsideNode.id))
+        verifyNoInteractions(nodeDB)
     }

-    @Test fun `getNodes fetches all nodes if none are cached`() {
+    @Test fun `getNodes fetches formerly cached nodes outside spatial cache after trim`() {
         val node1 = node(1, LatLon(0.0, 0.0))
         val node2 = node(2, LatLon(0.0001, 0.0001))
         val node3 = node(3, LatLon(0.0002, 0.0002))
         val node4 = node(4, LatLon(1.0, 1.0))
         val nodes = listOf(node1, node2, node3, node4)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = nodes)
+        cache.trim(4)

         val nodeDB: NodeDao = mock()
-        val nodeIds = nodes.map { it.id }.toHashSet() // use hashSet to avoid issues with order
+        val nodeIds = nodes.map { it.id }.sorted() // use sorted to avoid issues with order
         on(nodeDB.getAll(nodeIds)).thenReturn(nodes)
-        assertTrue(cache.getNodes(nodes.map { it.id }) { nodeDB.getAll(it.toHashSet()) }.containsExactlyInAnyOrder(nodes))
+        assertTrue(cache.getNodes(nodes.map { it.id }) { nodeDB.getAll(it.sorted()) }.containsExactlyInAnyOrder(nodes))
         verify(nodeDB).getAll(nodeIds)
     }

@@ -203,7 +222,7 @@ internal class MapDataCacheTest {
         val rel2 = rel(2)
         val elements = listOf(node1, way1, way2, way3, rel1, rel2)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(bbox = node1.position.enclosingTilePos(16).asBoundingBox(16))
         cache.update(updatedElements = elements)

@@ -220,7 +239,7 @@ internal class MapDataCacheTest {
         val elements = listOf(node1, way1, way2, way3, rel1, rel2)
         val cachedElements = listOf(way1, rel1)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = cachedElements)
         val keysNotInCache = elements.filterNot { it in cachedElements }.map { ElementKey(it.type, it.id) }.toHashSet()

@@ -229,10 +248,10 @@ internal class MapDataCacheTest {
         assertTrue(cache.getElements(elements.map { ElementKey(it.type, it.id) }) { elementDB.getAll(it.toHashSet()) }.containsExactlyInAnyOrder(elements))
         verify(elementDB).getAll(keysNotInCache)

-        // check whether elements except node1 are cached now
+        // check whether elements are cached now
         on(elementDB.getAll(listOf(ElementKey(node1.type, node1.id)))).thenReturn(listOf(node1))
         assertTrue(cache.getElements(elements.map { ElementKey(it.type, it.id) }) { elementDB.getAll(it) }.containsExactlyInAnyOrder(elements))
-        verify(elementDB).getAll(listOf(ElementKey(node1.type, node1.id)))
+        verifyNoMoreInteractions(elementDB)
     }

     @Test fun `getElements fetches all elements if none are cached`() {
@@ -244,7 +263,7 @@ internal class MapDataCacheTest {
         val rel2 = rel(2)
         val elements = listOf(node1, way1, way2, way3, rel1, rel2)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val keys = elements.map { ElementKey(it.type, it.id) }.toHashSet()

         val elementDB: ElementDao = mock()
@@ -252,10 +271,10 @@ internal class MapDataCacheTest {
         assertTrue(cache.getElements(elements.map { ElementKey(it.type, it.id) }) { elementDB.getAll(it.toHashSet()) }.containsExactlyInAnyOrder(elements))
         verify(elementDB).getAll(keys)

-        // check whether elements except node1 are cached now
+        // check whether elements are cached now
         on(elementDB.getAll(listOf(ElementKey(node1.type, node1.id)))).thenReturn(listOf(node1))
         assertTrue(cache.getElements(elements.map { ElementKey(it.type, it.id) }) { elementDB.getAll(it) }.containsExactlyInAnyOrder(elements))
-        verify(elementDB).getAll(listOf(ElementKey(node1.type, node1.id)))
+        verifyNoMoreInteractions(elementDB)
     }

     @Test fun `getGeometries doesn't fetch cached geometries`() {
@@ -269,7 +288,7 @@ internal class MapDataCacheTest {
         val nodeEntry = ElementGeometryEntry(ElementType.NODE, 1L, nodeGeo)
         val entries = listOf(entry1, entry2, nodeEntry)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(bbox = node.position.enclosingTilePos(16).asBoundingBox(16))
         cache.update(updatedElements = listOf(node), updatedGeometries = entries)

@@ -288,7 +307,7 @@ internal class MapDataCacheTest {
         val entries = listOf(entry1, entry2, nodeEntry)
         val cachedEntries = listOf(entry1)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedGeometries = cachedEntries)
         val keysNotInCache = entries.filterNot { it in cachedEntries }.map { ElementKey(it.elementType, it.elementId) }.toHashSet()

@@ -314,7 +333,7 @@ internal class MapDataCacheTest {
         val nodeEntry = ElementGeometryEntry(ElementType.NODE, 1L, nodeGeo)
         val entries = listOf(entry1, entry2, nodeEntry)

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         val keys = entries.map { ElementKey(it.elementType, it.elementId) }.toHashSet()

         val geometryDB: ElementGeometryDao = mock()
@@ -328,30 +347,6 @@ internal class MapDataCacheTest {
         verify(geometryDB).getAllEntries(listOf(ElementKey(node1.type, node1.id)))
     }

-    @Test fun `update only puts nodes if tile is cached`() {
-        val node = node(1, LatLon(0.0, 0.0))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
-        val nodeTile = node.position.enclosingTilePos(16)
-        assertEquals(0, cache.getMapDataWithGeometry(nodeTile.asBoundingBox(16)).size)
-        // now we have nodeTile cached
-        cache.update(updatedElements = listOf(node))
-        assertTrue(cache.getMapDataWithGeometry(nodeTile.asBoundingBox(16)).nodes.containsExactlyInAnyOrder(listOf(node)))
-        assertEquals(node, cache.getElement(ElementType.NODE, 1L) { _, _ -> null })
-
-        val otherNode = node(2, LatLon(0.0, 1.0))
-        val otherNodeTile = otherNode.position.enclosingTilePos(16)
-        cache.update(updatedElements = listOf(otherNode))
-        assertNull(cache.getElement(ElementType.NODE, 2L) { _, _ -> null })
-        assertTrue(cache.getMapDataWithGeometry(otherNodeTile.asBoundingBox(16)).nodes.isEmpty())
-
-        val movedNode = node(1, LatLon(1.0, 0.0))
-        val movedNodeTile = movedNode.position.enclosingTilePos(16)
-        cache.update(updatedElements = listOf(movedNode))
-        assertNull(cache.getElement(ElementType.NODE, 1L) { _, _ -> null })
-        assertTrue(cache.getMapDataWithGeometry(movedNodeTile.asBoundingBox(16)).nodes.isEmpty())
-        assertTrue(cache.getMapDataWithGeometry(nodeTile.asBoundingBox(16)).nodes.isEmpty())
-    }
-
     @Test fun `update removes elements`() {
         val node = node(1, LatLon(0.0, 0.0))
         val way = way(2)
@@ -359,7 +354,7 @@ internal class MapDataCacheTest {
         val nodeKey = ElementKey(ElementType.NODE, 1L)
         val wayKey = ElementKey(ElementType.WAY, 2L)
         val relationKey = ElementKey(ElementType.RELATION, 3L)
-        val cache = MapDataCache(16, 4, 10) { listOf(node) to emptyList() }
+        val cache = MapDataCache(16, 4, 10, { listOf(node) to emptyList() }, { emptyList() })
         cache.getMapDataWithGeometry(node.position.enclosingTilePos(16).asBoundingBox(16))
         cache.update(updatedElements = listOf(way, rel))
         assertTrue(
@@ -376,7 +371,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val way2 = way(2, nodes = listOf(3L, 1L))
         val way3 = way(3, nodes = listOf(3L, 2L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(way1, way2, way3))
         val wayDB: WayDao = mock()
         on(wayDB.getAllForNode(1L)).thenReturn(listOf(way1, way2)).thenThrow(IllegalStateException())
@@ -396,7 +391,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val way2 = way(2, nodes = listOf(3L, 1L))
         val way3 = way(3, nodes = listOf(3L, 2L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, node3, way1, way2, way3), bbox = nodesRect.asBoundingBox(16))

         assertTrue(cache.getWaysForNode(1L) { emptyList() }.containsExactlyInAnyOrder(listOf(way1, way2)))
@@ -413,7 +408,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val way2 = way(2, nodes = listOf(3L, 1L))
         val way3 = way(3, nodes = listOf(3L, 2L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, node3, way1, way2, way3), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getWaysForNode(1L) { emptyList() }.containsExactlyInAnyOrder(listOf(way1, way2)))

@@ -430,7 +425,7 @@ internal class MapDataCacheTest {
         assertTrue(nodesRect.size <= 4) // fits in cache
         val way1 = way(1, nodes = listOf(1L, 2L))
         val way2 = way(2, nodes = listOf(3L, 1L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, node3, way1, way2), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getWaysForNode(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(way1, way2)))
         assertTrue(cache.getWaysForNode(2L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(way1)))
@@ -450,7 +445,7 @@ internal class MapDataCacheTest {
         assertTrue(nodesRect.size <= 4) // fits in cache
         val way1 = way(1, nodes = listOf(1L, 2L))
         val way2 = way(2, nodes = listOf(3L, 1L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, node3, way1, way2), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getWaysForNode(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(way1, way2)))
         assertTrue(cache.getWaysForNode(2L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(way1)))
@@ -471,7 +466,7 @@ internal class MapDataCacheTest {
         assertTrue(nodesRect.size <= 4) // fits in cache
         val way1 = way(1, nodes = listOf(1L, 2L))
         val way2 = way(2, nodes = listOf(3L, 1L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, node3, way1, way2), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getWaysForNode(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(way1, way2)))
         assertTrue(cache.getWaysForNode(2L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(way1)))
@@ -492,7 +487,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1L, "")))
         val rel2 = rel(2, members = listOf(RelationMember(ElementType.WAY, 1L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, way1, rel1, rel2), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getRelationsForNode(1L) { emptyList() }.containsExactlyInAnyOrder(listOf(rel1)))
         assertTrue(cache.getRelationsForWay(1L) { emptyList() }.containsExactlyInAnyOrder(listOf(rel2)))
@@ -511,7 +506,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1L, "")))
         val rel2 = rel(2, members = listOf(RelationMember(ElementType.NODE, 1L, ""), RelationMember(ElementType.WAY, 1L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, way1, rel1, rel2), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getRelationsForNode(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(rel1, rel2)))
         assertTrue(cache.getRelationsForWay(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(rel2)))
@@ -529,7 +524,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1L, "")))
         val rel2 = rel(2, members = listOf(RelationMember(ElementType.WAY, 1L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, way1, rel1, rel2), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getRelationsForNode(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(rel1)))
         assertTrue(cache.getRelationsForWay(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(rel2)))
@@ -548,7 +543,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1L, ""), RelationMember(ElementType.WAY, 1L, "")))
         val rel2 = rel(2, members = listOf(RelationMember(ElementType.WAY, 1L, ""), RelationMember(ElementType.NODE, 1L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, way1, rel1, rel2), bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getRelationsForNode(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(rel1, rel2)))
         assertTrue(cache.getRelationsForWay(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(rel1, rel2)))
@@ -560,6 +555,38 @@ internal class MapDataCacheTest {
         assertTrue(cache.getRelationsForWay(1L) { throw IllegalStateException() }.containsExactlyInAnyOrder(listOf(rel1updated)))
     }

+    @Test fun `cache returns nodes outside the BBox if part of ways`() {
+        val node1 = node(1, LatLon(0.0, 0.0))
+        val node2 = node(2, LatLon(0.0001, 0.0001))
+        val node3 = node(3, LatLon(0.0002, 0.0002))
+        val nodes = listOf(node1, node2, node3)
+        val outsideNode = node(4, LatLon(1.0, 1.0))
+        val fullyOutsideNode = node(5, LatLon(2.0, 2.0))
+        val nodesRect = listOf(node1.position, node2.position, node3.position).enclosingBoundingBox().enclosingTilesRect(16)
+        assertTrue(nodesRect.size <= 4) // fits in cache
+
+        val way1 = way(1, nodes = listOf(1L, 2L))
+        val way2 = way(2, nodes = listOf(3L, 1L))
+        val way3 = way(3, nodes = listOf(3L, 2L, 4L))
+        val ways = listOf(way1, way2, way3)
+        val outsideWay = way(4, nodes = listOf(4L, 5L))
+        val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1L, ""), RelationMember(ElementType.WAY, 5L, "")))
+        val outsideRel = rel(2, members = listOf(RelationMember(ElementType.NODE, 4L, ""), RelationMember(ElementType.WAY, 5L, "")))
+        val cache = getEmptyMapDataCache()
+        cache.update(updatedElements = nodes + ways + listOf(outsideNode, fullyOutsideNode, outsideWay, rel1, outsideRel), bbox = nodesRect.asBoundingBox(16))
+
+        // outsideNode is expected because it's part of returned way1
+        val expectedMapData = MutableMapDataWithGeometry(nodes + ways + rel1 + outsideNode, emptyList())
+        // node geometries get created when getting the data
+        nodes.forEach { expectedMapData.putGeometry(it.type, it.id, ElementPointGeometry(it.position)) }
+        expectedMapData.putGeometry(outsideNode.type, outsideNode.id, ElementPointGeometry(outsideNode.position))
+        // way and relation geometries are also put, though null
+        ways.forEach { expectedMapData.putGeometry(it.type, it.id, null) }
+        expectedMapData.putGeometry(rel1.type, rel1.id, null)
+        expectedMapData.boundingBox = nodesRect.asBoundingBox(16)
+        assertEquals(expectedMapData, cache.getMapDataWithGeometry(nodesRect.asBoundingBox(16)))
+    }
+
     @Test fun `trim removes everything not referenced by spatialCache`() {
         val node1 = node(1, LatLon(0.0, 0.0))
         val node2 = node(2, LatLon(0.0001, 0.0001))
@@ -576,25 +603,29 @@ internal class MapDataCacheTest {
         val outsideWay = way(4, nodes = listOf(4L, 5L))
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1L, ""), RelationMember(ElementType.WAY, 5L, "")))
         val outsideRel = rel(2, members = listOf(RelationMember(ElementType.NODE, 4L, ""), RelationMember(ElementType.WAY, 5L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = nodes + ways + listOf(outsideNode, outsideWay, rel1, outsideRel), bbox = nodesRect.asBoundingBox(16))

-        val expectedMapData = MutableMapDataWithGeometry(nodes + ways + rel1, emptyList())
+        // outsideNode is expected because it's part of returned way1
+        val expectedMapData = MutableMapDataWithGeometry(nodes + ways + rel1 + outsideNode, emptyList())
         // node geometries get created when getting the data
         nodes.forEach { expectedMapData.putGeometry(it.type, it.id, ElementPointGeometry(it.position)) }
+        expectedMapData.putGeometry(outsideNode.type, outsideNode.id, ElementPointGeometry(outsideNode.position))
         // way and relation geometries are also put, though null
         ways.forEach { expectedMapData.putGeometry(it.type, it.id, null) }
         expectedMapData.putGeometry(rel1.type, rel1.id, null)
         expectedMapData.boundingBox = nodesRect.asBoundingBox(16)
         assertEquals(expectedMapData, cache.getMapDataWithGeometry(nodesRect.asBoundingBox(16)))

-        assertEquals(null, cache.getElement(ElementType.NODE, 4L) { _, _ -> null }) // node not in spatialCache
+        assertEquals(outsideNode, cache.getElement(ElementType.NODE, 4L) { _, _ -> null }) // node in nodeCache
         assertEquals(outsideWay, cache.getElement(ElementType.WAY, 4L) { _, _ -> null })
         assertEquals(outsideRel, cache.getElement(ElementType.RELATION, 2L) { _, _ -> null })
         cache.trim(4)
-        // outside way and relation removed
+        // outside node, way and relation removed
+        assertEquals(null, cache.getElement(ElementType.NODE, 4L) { _, _ -> null })
         assertEquals(null, cache.getElement(ElementType.WAY, 4L) { _, _ -> null })
         assertEquals(null, cache.getElement(ElementType.RELATION, 2L) { _, _ -> null })
+        // but relation partially in spatial cache area is not removed
         assertEquals(rel1, cache.getElement(ElementType.RELATION, 1L) { _, _ -> null })
     }

@@ -603,7 +634,7 @@ internal class MapDataCacheTest {
         val nodeRectBbox = node.position.enclosingBoundingBox(0.1).asBoundingBoxOfEnclosingTiles(16)

         val way = way(1, nodes = listOf(1L, 2L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(way, node), bbox = nodeRectBbox)

         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1, "")))
@@ -622,7 +653,7 @@ internal class MapDataCacheTest {
         val way1 = way(1, nodes = listOf(1L, 2L))
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 1L, "")))
         val rel2 = rel(2, members = listOf(RelationMember(ElementType.WAY, 1L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, way1, rel1, rel2), bbox = nodesRect.asBoundingBox(16))

         // not empty
@@ -634,7 +665,7 @@ internal class MapDataCacheTest {

     @Test fun `update doesn't create waysByNodeId entry if node is not in spatialCache`() {
         val way1 = way(1, nodes = listOf(1L, 2L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(way1))

         val wayDB: WayDao = mock()
@@ -655,7 +686,7 @@ internal class MapDataCacheTest {
         val nodesRect = listOf(node1.position, node2.position, node3.position).enclosingBoundingBox().enclosingTilesRect(16)
         assertTrue(nodesRect.size <= 4) // fits in cache

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = nodes, bbox = nodesRect.asBoundingBox(16))

         val way1 = way(1, nodes = listOf(1L, 2L))
@@ -667,7 +698,7 @@ internal class MapDataCacheTest {

     @Test fun `update does add way to waysByNodeId entry if entry already exists`() {
         val way1 = way(1, nodes = listOf(1L, 2L))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(way1))

         val wayDB: WayDao = mock()
@@ -689,7 +720,7 @@ internal class MapDataCacheTest {
         val node2 = node(2)
         val way1 = way(1, nodes = listOf(1L))
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.NODE, 2L, ""), RelationMember(ElementType.WAY, 1L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(node1, node2, way1, rel1))

         val relationDB: RelationDao = mock()
@@ -709,7 +740,7 @@ internal class MapDataCacheTest {
         val nodesRect = listOf(node1.position, node2.position, node3.position).enclosingBoundingBox().enclosingTilesRect(16)
         assertTrue(nodesRect.size <= 4) // fits in cache

-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = nodes, bbox = nodesRect.asBoundingBox(16))

         val way1 = way(1, nodes = listOf(1L, 2L))
@@ -722,7 +753,7 @@ internal class MapDataCacheTest {

     @Test fun `update does add way to relationsByElementKey entry if entry already exists`() {
         val rel1 = rel(1, members = listOf(RelationMember(ElementType.WAY, 1L, "")))
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()
         cache.update(updatedElements = listOf(rel1))

         val relationDB: RelationDao = mock()
@@ -754,7 +785,7 @@ internal class MapDataCacheTest {
         val elementsInsideBBox = listOf(node1, node2, node3, way1, way2, way3, rel1, rel2)
         val elementDB: ElementDao = mock()
         on(elementDB.getAll(nodesRect.asBoundingBox(16))).thenReturn(elementsInsideBBox)
-        val cache = MapDataCache(16, 4, 10) { elementDB.getAll(it) to emptyList() }
+        val cache = MapDataCache(16, 4, 10, { elementDB.getAll(it) to emptyList() }, { emptyList() })

         val expectedMapData = MutableMapDataWithGeometry().apply {
             listOf(node1, node2, node3).forEach { put(it, ElementPointGeometry(it.position)) }
@@ -784,7 +815,7 @@ internal class MapDataCacheTest {
         val rel2 = rel(2, members = listOf(RelationMember(ElementType.WAY, 1L, "")))
         val elementDB: ElementDao = mock()
         on(elementDB.getAll(node1rect.asBoundingBox(16))).thenReturn(listOf(node1, node3, way2, way3, rel1))
-        val cache = MapDataCache(16, 4, 10) { elementDB.getAll(it) to emptyList() }
+        val cache = MapDataCache(16, 4, 10, { elementDB.getAll(it) to emptyList() }, { emptyList() })

         cache.update(updatedElements = listOf(node2, way1, rel2), bbox = node2rect.asBoundingBox(16))

@@ -816,7 +847,7 @@ internal class MapDataCacheTest {
         val outsideNode = node(4, LatLon(1.0, 1.0))
         val outsideWay = way(4)
         val outsideRel = rel(3)
-        val cache = MapDataCache(16, 4, 10) { emptyList<Element>() to emptyList() }
+        val cache = getEmptyMapDataCache()

         cache.update(updatedElements = elementsInsideBBox + outsideNode + outsideWay + outsideRel, bbox = nodesRect.asBoundingBox(16))
         assertTrue(cache.getMapDataWithGeometry(nodesBBox).toList().containsExactlyInAnyOrder(elementsInsideBBox))
@@ -830,7 +861,7 @@ internal class MapDataCacheTest {
         val rect2 = TilesRect(x, y + 1, x, y + 2).asBoundingBox(16)
         val node1 = node(1, LatLon(rect.min.latitude + 0.00001, rect.min.longitude + 0.00001)) // node in rect and rect2
         val node2 = node(2, LatLon(rect.max.latitude - 0.00001, rect.max.longitude - 0.00001)) // node in rect only
-        val cache = MapDataCache(16, 24, 10) {
+        val cache = MapDataCache(16, 24, 10, {
             val elements = when {
                 it == rect -> listOf(node1, node2) // may not be true if the bbox is extended because of the tilesRect issue (happens for x=2, y=5, but not for x=2, y=1)
                 rect.isCompletelyInside(it) -> listOf(node1, node2) // fix for above
@@ -838,7 +869,7 @@ internal class MapDataCacheTest {
                 else -> emptyList()
             }
             elements to emptyList()
-        }
+        }, { emptyList() })
         // fill cache
         assertTrue(cache.getMapDataWithGeometry(rect).toList().containsExactlyInAnyOrder(listOf(node1, node2)))

@@ -849,3 +880,5 @@ internal class MapDataCacheTest {
         assertTrue(cache.getMapDataWithGeometry(rect).toList().containsExactlyInAnyOrder(listOf(node1, node2)))
     }
 }
+
+private fun getEmptyMapDataCache() = MapDataCache(16, 4, 10, { emptyList<Element>() to emptyList() }, { emptyList() })


Fix #4790

@Helium314 are you interested in reviewing this?

Cycleway quest applies wrong tags

Cycleway quest applies wrong tags
Likely this is related to #4774, but this happens in in 51.0 beta1 (from GitHub)

* Open the cycleway quest for https://www.openstreetmap.org/way/43943837 (`cycleway=lane`)
* see current situation tagged as "???" (which is expected)
* either: set the available side to "none"
  * see added tags `oneway:bicycle=no`, `cycleway:left=lane`, `cycleway:right=no` (the first two are incorrect)
* or: set the available side to "advisory"
  * see added tags `oneway:bicycle=no`, `cycleway:both=lane`, `cycleway:right:lane=advisory` (the first two are incorrect)
* get the quest again, this time showing both sides
* set new side to "none"
* see tags are now correct

Uh, this one is a bugger.

`cycleway=lane` on a `oneway=yes` is interpreted to mean

`cycleway:right=lane`
`cycleway:left` not set (`null`)

So, as the left side is `null`, the left side is also not shown in the UI.

However, as only the right side is **changed** by the user, only `cycleway:right=lane` + `cycleway:right:lane=advisory` is set and whatever the value that was in `cycleway` tag before is now updated to only apply for the left side. This is why we end up with `cycleway:left=lane`.

Probably will need to change the logic that expands `cycleway` into `cycleway:left` and `cycleway:right` in `CyclewayCreator.kt`. Not sure yet what's the cleanest way to do this.

./gradlew :app:testDebugUnitTest --tests "de.westnordost.streetcomplete.osm.cycleway.CyclewayCreatorKtTest"

diff --git a/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreator.kt b/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreator.kt
index dbee23c4259..de54fe90137 100644
--- a/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreator.kt
+++ b/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreator.kt
@@ -7,6 +7,7 @@ import de.westnordost.streetcomplete.osm.expandSides
 import de.westnordost.streetcomplete.osm.hasCheckDateForKey
 import de.westnordost.streetcomplete.osm.isInContraflowOfOneway
 import de.westnordost.streetcomplete.osm.isOneway
+import de.westnordost.streetcomplete.osm.isReversedOneway
 import de.westnordost.streetcomplete.osm.mergeSides
 import de.westnordost.streetcomplete.osm.sidewalk.LeftAndRightSidewalk
 import de.westnordost.streetcomplete.osm.sidewalk.Sidewalk
@@ -22,10 +23,12 @@ fun LeftAndRightCycleway.applyTo(tags: Tags, isLeftHandTraffic: Boolean) {
        - cycleway:left=separate
        First separating the values and then later conflating them again, if possible, solves this.
      */
-    tags.expandSides("cycleway")
-    tags.expandSides("cycleway", "lane")
-    tags.expandSides("cycleway", "oneway")
-    tags.expandSides("cycleway", "segregated")
+
+    expandBareTags(tags, isLeftHandTraffic)
+    tags.expandSides("cycleway", null, false)
+    tags.expandSides("cycleway", "lane", false)
+    tags.expandSides("cycleway", "oneway", false)
+    tags.expandSides("cycleway", "segregated", false)

     applyOnewayNotForCyclists(tags, isLeftHandTraffic)
     left?.applyTo(tags, false, isLeftHandTraffic)
@@ -49,6 +52,36 @@ fun LeftAndRightCycleway.applyTo(tags: Tags, isLeftHandTraffic: Boolean) {
     ).applyTo(tags)
 }

+/* bare cycleway tags are interpreted differently for oneways */
+private fun expandBareTags(tags: Tags, isLeftHandTraffic: Boolean) {
+    val cycleway = tags["cycleway"] ?: return
+    // i.e. they are only expanded into one side. Which side depends on country, direction of oneway
+    // and whether it is an "opposite" tag value
+    val side = if (isOneway(tags)) {
+        val isReverseSideRight = isReversedOneway(tags) xor isLeftHandTraffic
+        val isOpposite = cycleway.startsWith("opposite")
+        if (isOpposite == isReverseSideRight) "right" else "left"
+    } else {
+        "both"
+    }
+    if (!tags.containsKey("cycleway:$side")) {
+        tags["cycleway:$side"] = cycleway
+            .removePrefix("opposite_") // opposite_track -> track etc.
+            .replaceFirst("opposite", "no") // opposite -> no
+    }
+    tags.remove("cycleway")
+    tags.expandBareTagIntoSide("cycleway", "lane", side)
+    tags.expandBareTagIntoSide("cycleway", "oneway", side)
+    tags.expandBareTagIntoSide("cycleway", "segregated", side)
+}
+
+private fun Tags.expandBareTagIntoSide(key: String, suffix: String = "", side: String) {
+    val post = if (suffix.isEmpty()) "" else ":$suffix"
+    val value = get("$key$post") ?: return
+    if (!containsKey("$key:$side$post")) set("$key:$side$post", value)
+    remove("$key$post")
+}
+
 private fun LeftAndRightCycleway.applyOnewayNotForCyclists(tags: Tags, isLeftHandTraffic: Boolean) {
     if (isOneway(tags) && isNotOnewayForCyclistsNow(tags, isLeftHandTraffic)) {
         tags["oneway:bicycle"] = "no"
diff --git a/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayParser.kt b/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayParser.kt
index 06770f67c6b..d71f4ed6220 100644
--- a/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayParser.kt
+++ b/app/src/main/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayParser.kt
@@ -38,8 +38,11 @@ fun createCyclewaySides(tags: Map<String, String>, isLeftHandTraffic: Boolean):
     *  it that way. */
     val nakedCycleway = createCyclewayForSide(expandedTags, null)
     if (isOneway && nakedCycleway != null && nakedCycleway != NONE) {
-        val isRight = if (isOpposite) isReverseSideRight else !isReverseSideRight
-        if (isRight) right = nakedCycleway else left = nakedCycleway
+        if (isOpposite == isReverseSideRight) {
+            right = nakedCycleway
+        } else {
+            left = nakedCycleway
+        }
     } else {
         left = createCyclewayForSide(expandedTags, false)
         right = createCyclewayForSide(expandedTags, true)

diff --git a/app/src/test/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreatorKtTest.kt b/app/src/test/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreatorKtTest.kt
index bf195940931..568eecd1d27 100644
--- a/app/src/test/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreatorKtTest.kt
+++ b/app/src/test/java/de/westnordost/streetcomplete/osm/cycleway/CyclewayCreatorKtTest.kt
@@ -697,6 +697,75 @@ class CyclewayCreatorKtTest {
         )
     }

+    @Test fun `apply answer for one side in oneway when bare tag was set before`() {
+        verifyAnswer(
+            mapOf("oneway" to "yes", "cycleway" to "track"),
+            cycleway(null, NONE),
+            arrayOf(
+                StringMapEntryDelete("cycleway", "track"),
+                StringMapEntryAdd("cycleway:right", "no")
+            )
+        )
+        verifyAnswer(
+            mapOf("oneway" to "-1", "cycleway" to "opposite"),
+            cycleway(TRACK, null),
+            arrayOf(
+                StringMapEntryDelete("cycleway", "opposite"),
+                StringMapEntryAdd("cycleway:right", "no"),
+                StringMapEntryAdd("cycleway:left", "track")
+            )
+        )
+        verifyAnswer(
+            mapOf(
+                "oneway" to "yes",
+                "cycleway" to "opposite_track",
+                "cycleway:segregated" to "yes"
+            ),
+            cycleway(null, NONE),
+            arrayOf(
+                StringMapEntryDelete("cycleway", "opposite_track"),
+                StringMapEntryDelete("cycleway:segregated", "yes"),
+                StringMapEntryAdd("cycleway:left", "track"),
+                StringMapEntryAdd("cycleway:left:segregated", "yes"),
+                StringMapEntryAdd("cycleway:right", "no"),
+                StringMapEntryAdd("oneway:bicycle", "no")
+            )
+        )
+        verifyAnswer(
+            mapOf(
+                "oneway" to "yes",
+                "cycleway" to "opposite_lane",
+                "cycleway:lane" to "advisory",
+                "cycleway:oneway" to "yes"
+            ),
+            cycleway(null, NONE),
+            arrayOf(
+                StringMapEntryDelete("cycleway", "opposite_lane"),
+                StringMapEntryDelete("cycleway:lane", "advisory"),
+                StringMapEntryDelete("cycleway:oneway", "yes"),
+                StringMapEntryAdd("cycleway:left", "lane"),
+                StringMapEntryAdd("cycleway:left:lane", "advisory"),
+                StringMapEntryAdd("cycleway:left:oneway", "yes"),
+                StringMapEntryAdd("cycleway:right", "no")
+            )
+        )
+    }
+
+    @Test fun `expanding bare tags does not overwrite non-bare tags`() {
+        verifyAnswer(
+            mapOf(
+                "oneway" to "yes",
+                "cycleway" to "track",
+                "cycleway:right" to "blubber",
+            ),
+            cycleway(NONE, null),
+            arrayOf(
+                StringMapEntryDelete("cycleway", "track"),
+                StringMapEntryAdd("cycleway:left", "no")
+            )
+        )
+    }
+
     @Test(expected = IllegalArgumentException::class)
     fun `applying invalid left throws exception`() {
         cycleway(INVALID, null).applyTo(StringMapChangesBuilder(mapOf()), false)