HTML sanitizer updates

Fixes #6240
Fixes #6241

<a> "name" attribute is sanitized

align=center is sanitized

<a> "name" attribute is sanitized
### Checklist

- [X] I have used the search function to see if someone else has already submitted the same bug report.
- [X] I will describe the problem with as much detail as possible.

### App version

6.202

### Where did you get the app from?

Other

### Android version

Havoc OS 4 (Android 11)

### Device model

Oneplus 3T

### Steps to reproduce

teltarif.de sends newsletters with a table of contents at the top.

In Thunderbird I can click on these headlines and it forwards me to the corresponding article/section down below.

K9 currently sanitizes name attributes of links though, so the webview(?) component doesn't know where to move the viewport to (no link with a corresponding name attribute below)

`<A NAME="a1"></A>`
is sanitized to
`<a></a>`

The corresponding link in the table of contents looks like this:
`<a href="#a1">`
You've probably seen countless websites like wikipedia who use this feature as well.
usually they use the id of an element though not an actual <a> tag with a name attribute, which is obsolete in newer html versions, I think (not that it matters for mails)

### Expected behavior

K9 shouldn't remove the attribute

### Actual behavior

K9 removes the attribute

### Logs

_No response_
align=center is sanitized
### Checklist

- [X] I have used the search function to see if someone else has already submitted the same bug report.
- [X] I will describe the problem with as much detail as possible.

### App version

6.202

### Where did you get the app from?

Other

### Android version

Havoc OS 4 (Android 11)

### Device model

Oneplus 3T

### Steps to reproduce

`<div class="alignment" align="center">`
is sanitized by the html cleaner code to
`<div class="alignment">`

(I verified this using the html-cleaner cli)

The div with "align center" is used to center a link in the mail (<a> that looks like a button)

I attached screenshots in the following sections to showcase the behaviour

### Expected behavior

![grafik](https://user-images.githubusercontent.com/5685512/184463886-ea762804-2fd9-41c6-b75d-8cfc2478ea1e.png)

### Actual behavior

![grafik](https://user-images.githubusercontent.com/5685512/184463905-7b3f939d-b84b-4950-a2bb-8fa0fd1ae698.png)

### Logs

_No response_

./gradlew :app:html-cleaner:test --tests "app.k9mail.html.cleaner.HtmlSanitizerTest"

diff --git a/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt b/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt
index 80fa043e840..5697b115c4f 100644
--- a/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt
+++ b/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt
@@ -15,6 +15,8 @@ internal class BodyCleaner {
         val allowList = Safelist.relaxed()
             .addTags("font", "hr", "ins", "del", "center", "map", "area", "title")
             .addAttributes("font", "color", "face", "size")
+            .addAttributes("a", "name")
+            .addAttributes("div", "align")
             .addAttributes(
                 "table", "align", "background", "bgcolor", "border", "cellpadding", "cellspacing",
                 "width"

diff --git a/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt b/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt
index 12ea9e797b9..8720a3273d5 100644
--- a/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt
+++ b/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt
@@ -409,6 +409,42 @@ class HtmlSanitizerTest {
         )
     }

+    @Test
+    fun `should keep 'align' attribute on 'div' element`() {
+        val html = """
text
"""
+
+        val result = htmlSanitizer.sanitize(html)
+
+        assertThat(result.toCompactString()).isEqualTo(
+            """
+
+
+
+
text

+
+
+            """.trimIndent().trimLineBreaks()
+        )
+    }
+
+    @Test
+    fun `should keep 'name' attribute on 'a' element`() {
+        val html = """"""
+
+        val result = htmlSanitizer.sanitize(html)
+
+        assertThat(result.toCompactString()).isEqualTo(
+            """
+
+
+
+
+
+
+            """.trimIndent().trimLineBreaks()
+        )
+    }
+
     private fun Document.toCompactString(): String {
         outputSettings()
             .prettyPrint(false)


Fix removing multiple notifications at once

Fixes #6077

Duplicate notifications for new messages
### Checklist

- [X] I have used the search function to see if someone else has already submitted the same bug report.
- [X] I will describe the problem with as much detail as possible.

### App version

6.000

### Where did you get the app from?

Google Play

### Android version

11

### Device model

Moto g10

### Steps to reproduce

Part of the trigger seems to be only closing the notifications about a few of the received mails and the  receiving more mails. More I didn't find out in the last months.

### Expected behavior

If n mails arrive I normally get notifications with all their title lines

### Actual behavior

Rarely I get n notifications with the title lone of one of my mails instead

### Logs

_No response_

![Screenshot_20220519-111601](https://user-images.githubusercontent.com/5041537/169515974-17934db9-97d2-4509-8b59-b181aa1ade81.png)

I actually got 8 mails, bit only one was about wxMaxima
This also happens to me sometimes, a bunch of duplicate notifications for the same email. I am using K-9 Mail 6.0 on GrapheneOS (Android 12) on a Pixel 4a.

This person has also noticed the issue: https://www.reddit.com/r/fdroid/comments/vob87m/why_does_k9mail_keep_sending_me_multiple/
Same problem. Driving me crazy!
On Android 11 the contents of the "collapsed notification" is generated by the OS. It usually takes the titles of the individual notifications to generate the summary.

Does everything look okay when you expand the notification to see the individual notifications? If that's the case, this might be a bug in your Android version.
Unfortunately this time it seems not to be a bug in this specific Android11 feature:
![Screenshot_20220709-114305](https://user-images.githubusercontent.com/5041537/178101951-0a372527-ef71-4e59-9de7-02b366bcbd89.png)

Issue is still present in version 6.20
I have the impression that a trigger might be that a reply arrives to an unread message that is dropped out of the mails kept in K9 by the arrival of the new message.
When this happens again, please try the following:

1. Force stop the app. This should also remove all notifications.
2. Do something that will start the app again. That will recreate the notifications from K-9 Mail's database. However, don't just open the app if this will open a message list view that contains the messages in the notifications (because that will lead to the notifications being removed). You could add a new "K-9 Unread" widget to the home screen. That process will launch the app, triggering the notifications to be recreated. But it won't display a message list that will lead to the notifications being removed.
3. Check if there are still duplicate notifications and report back.
We are lucky: Normally this triggers about once per week. Today I found two more puzzle pieces: I got 6 repeated notifications and one notification about a different mail.
And killing K9 and restarting it using the widget caused only the repeated notifications to reappear.
![Screenshot_20220710-080401](https://user-images.githubusercontent.com/5041537/178133664-633c0e43-0893-4811-b792-a4f3f2652134.png)

That result is raising more questions than it is able to provide answers.

So you started out with 7 notifications, 6 of them for the same message? And you ended up with 3 notifications, all of them for the same message, the one that was repeated 6 times before the app restart?

What happened to the other message? Was it deleted, moved, or marked as read using another client?

Before I made the screenshot the 6 repeated Mails were still displayed in the notification drawer. I believed that the other mail that was no more shown might have been marked as read by tapping on the K9 widget. My kmail wasn't in a working order when I made this screenshot...
Ok. So it sounds like all duplicate notifications are being restored. The database table that stores information about notifications doesn't allow more than one entry per internal message ID. That suggests that for K-9 Mail all of these entries are distinct messages.

When this happens again, please do the following:
1. Tap an individual notification to open the message. That should open the message view and remove the notification for that message.
2. Don't go back to the message list screen (because that would remove all notifications for messages in that folder). Leave the app using the home button.

Does that remove all duplicate notifications or just the one you tapped?

Repeat the process with the other notifications. Does tapping one of them remove more than one notification?

It removes only the duplicate I tried to remove. As if I get 10 duplicates that means I got 10 messages I guess that already made sense.

...but going back to the message list screen doesn't remove the duplicate notifications for me. As it often doesn't do so for non-duplicate messages that might possibly not mean anything, though...
@gunterkoenigsmann, do you pull/push multiple folders that contain the same message?

I had a similar problem with gmail, where k9mail gave duplicate notifications for all received messages.
I disabled pull/push for the Gmail/All-mail folder, so that I am pushing/polling just my inbox folder.
And now I get just one notification.
Maybe the title is misleading now: I actually get one notification per message. But sometimes (very rarely) many of the notifications get the notification text from the same message.

./gradlew :app:core:testDebugUnitTest --tests "com.fsck.k9.notification.NotificationDataStoreTest"

diff --git a/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt b/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt
index db99f40340e..de9f8c94bea 100644
--- a/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt
+++ b/app/core/src/main/java/com/fsck/k9/notification/NotificationDataStore.kt
@@ -108,32 +108,43 @@ internal class NotificationDataStore {
         account: Account,
         selector: (List<MessageReference>) -> List<MessageReference>
     ): RemoveNotificationsResult? {
-        val notificationData = getNotificationData(account)
+        var notificationData = getNotificationData(account)
         if (notificationData.isEmpty()) return null

         val removeMessageReferences = selector.invoke(notificationData.messageReferences)
+        if (removeMessageReferences.isEmpty()) return null

         val operations = mutableListOf<NotificationStoreOperation>()
         val newNotificationHolders = mutableListOf<NotificationHolder>()
         val cancelNotificationIds = mutableListOf<Int>()

-        for (messageReference in removeMessageReferences) {
-            val notificationHolder = notificationData.activeNotifications.firstOrNull {
-                it.content.messageReference == messageReference
+        val activeMessageReferences = notificationData.activeNotifications.map { it.content.messageReference }.toSet()
+        val (removeActiveMessageReferences, removeInactiveMessageReferences) = removeMessageReferences
+            .partition { it in activeMessageReferences }
+
+        if (removeInactiveMessageReferences.isNotEmpty()) {
+            val inactiveMessageReferences = notificationData.inactiveNotifications
+                .map { it.content.messageReference }.toSet()
+
+            for (messageReference in removeInactiveMessageReferences) {
+                if (messageReference in inactiveMessageReferences) {
+                    operations.add(NotificationStoreOperation.Remove(messageReference))
+                }
             }

-            if (notificationHolder == null) {
-                val inactiveNotificationHolder = notificationData.inactiveNotifications.firstOrNull {
-                    it.content.messageReference == messageReference
-                } ?: continue
+            val removeMessageReferenceSet = removeInactiveMessageReferences.toSet()
+            notificationData = notificationData.copy(
+                inactiveNotifications = notificationData.inactiveNotifications
+                    .filter { it.content.messageReference !in removeMessageReferenceSet }
+            )
+        }

-                operations.add(NotificationStoreOperation.Remove(messageReference))
+        for (messageReference in removeActiveMessageReferences) {
+            val notificationHolder = notificationData.activeNotifications.first {
+                it.content.messageReference == messageReference
+            }

-                val newNotificationData = notificationData.copy(
-                    inactiveNotifications = notificationData.inactiveNotifications - inactiveNotificationHolder
-                )
-                notificationDataMap[account.uuid] = newNotificationData
-            } else if (notificationData.inactiveNotifications.isNotEmpty()) {
+            if (notificationData.inactiveNotifications.isNotEmpty()) {
                 val newNotificationHolder = notificationData.inactiveNotifications.first()
                     .toNotificationHolder(notificationHolder.notificationId)

@@ -148,29 +159,29 @@ internal class NotificationDataStore {
                     )
                 )

-                val newNotificationData = notificationData.copy(
+                notificationData = notificationData.copy(
                     activeNotifications = notificationData.activeNotifications - notificationHolder +
                         newNotificationHolder,
                     inactiveNotifications = notificationData.inactiveNotifications.drop(1)
                 )
-                notificationDataMap[account.uuid] = newNotificationData
             } else {
                 cancelNotificationIds.add(notificationHolder.notificationId)

                 operations.add(NotificationStoreOperation.Remove(messageReference))

-                val newNotificationData = notificationData.copy(
+                notificationData = notificationData.copy(
                     activeNotifications = notificationData.activeNotifications - notificationHolder
                 )
-                notificationDataMap[account.uuid] = newNotificationData
             }
         }

+        notificationDataMap[account.uuid] = notificationData
+
         return if (operations.isEmpty()) {
             null
         } else {
             RemoveNotificationsResult(
-                notificationData = getNotificationData(account),
+                notificationData = notificationData,
                 notificationStoreOperations = operations,
                 notificationHolders = newNotificationHolders,
                 cancelNotificationIds = cancelNotificationIds

diff --git a/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt b/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt
index 57ffc8aba4f..c3000f77111 100644
--- a/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt
+++ b/app/core/src/test/java/com/fsck/k9/notification/NotificationDataStoreTest.kt
@@ -92,6 +92,38 @@ class NotificationDataStoreTest : RobolectricTest() {
         }
     }

+    @Test
+    fun `remove multiple notifications`() {
+        repeat(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1) { index ->
+            notificationDataStore.addNotification(account, createNotificationContent(index.toString()), TIMESTAMP)
+        }
+
+        val result = notificationDataStore.removeNotifications(account) { it.dropLast(1) }
+
+        assertNotNull(result) { removeResult ->
+            assertThat(removeResult.notificationData.newMessagesCount).isEqualTo(1)
+            assertThat(removeResult.cancelNotificationIds).hasSize(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS)
+        }
+    }
+
+    @Test
+    fun `remove all notifications`() {
+        repeat(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1) { index ->
+            notificationDataStore.addNotification(account, createNotificationContent(index.toString()), TIMESTAMP)
+        }
+
+        val result = notificationDataStore.removeNotifications(account) { it }
+
+        assertNotNull(result) { removeResult ->
+            assertThat(removeResult.notificationData.newMessagesCount).isEqualTo(0)
+            assertThat(removeResult.notificationHolders).hasSize(0)
+            assertThat(removeResult.notificationStoreOperations).hasSize(MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1)
+            for (notificationStoreOperation in removeResult.notificationStoreOperations) {
+                assertThat(notificationStoreOperation).isInstanceOf(NotificationStoreOperation.Remove::class.java)
+            }
+        }
+    }
+
     @Test
     fun testRemoveDoesNotLeakNotificationIds() {
         for (i in 1..MAX_NUMBER_OF_NEW_MESSAGE_NOTIFICATIONS + 1) {


Allow all URI schemes in links

Fixes #6103

sms: link not working

sms: link not working
### Checklist

- [x] I have used the search function to see if someone else has already submitted the same bug report.
- [X] I will describe the problem with as much detail as possible.

### App version

6.000

### Where did you get the app from?

Google Play

### Android version

7.1.1

### Device model

E5823

### Steps to reproduce

1. Go to a mail with a "sms:" link
2. Click on them
3. Nothing appears

NOTE :

I've tested this on 2 different **android** smartphones. Both have this problem.

### Expected behavior

In others (mobile) mail clients, this link is opening the SMS interface to send a specific SMS to someone.

### Actual behavior

Nothing. The click action have any effect.

### Logs

_No response_

I confirm.
It seams to be a "parsing issue", links built with a sms: URI are not recognized "as links" so they are not clickable.
Tested forms :

<a href="sms:+0102030405&body=Hello" ...
<a href="sms:?body=Hello" ...
<a href="sms:&body=Hello" ...
<a href="sms:?&body=Hello" ...
<a href="sms:/?&body=Hello" ...

diff --git a/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt b/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt
index 327d7f80613..80fa043e840 100644
--- a/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt
+++ b/app/html-cleaner/src/main/java/app/k9mail/html/cleaner/BodyCleaner.kt
@@ -36,7 +36,9 @@ internal class BodyCleaner {
             .addAttributes("img", "usemap")
             .addAttributes(":all", "class", "style", "id", "dir")
             .addProtocols("img", "src", "http", "https", "cid", "data")
-            .addProtocols("a", "href", "tel", "sip", "bitcoin", "ethereum", "rtsp")
+
+            // Allow all URI schemes in links
+            .removeProtocols("a", "href", "ftp", "http", "https", "mailto")

         cleaner = Cleaner(allowList)
     }

diff --git a/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt b/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt
index 9ebc3857b5e..12ea9e797b9 100644
--- a/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt
+++ b/app/html-cleaner/src/test/java/app/k9mail/html/cleaner/HtmlSanitizerTest.kt
@@ -301,6 +301,8 @@ class HtmlSanitizerTest {
         )
     }

+    // This test will fail when jsoup updates its list of allowed "protocols" for the a.href attribute.
+    // When that happens, please adjust the removeProtocols("a", "href", …) line in BodyCleaner.
     @Test
     fun shouldKeepUris() {
         val html =
@@ -311,9 +313,9 @@ class HtmlSanitizerTest {
             HTTPS
             Mailto
             Telephone
+            SMS
             SIP
-            Bitcoin
-            Ethereum
+            Unknown
             RTSP

@@ -330,9 +332,9 @@ class HtmlSanitizerTest {
             HTTPS
             Mailto
             Telephone
+            SMS
             SIP
-            Bitcoin
-            Ethereum
+            Unknown
             RTSP