Commit Graph

871 Commits

Author SHA1 Message Date
3353062dbe webmail: when moving out all messages in a thread (none remaining in view), don't cause js error but select next message
removing an item from the selected list should be done regardless of focus,
i.e. the code snippet shouldn't have been behind the "if (focus...)" condition.
2023-09-22 14:25:25 +02:00
be5f804d5b webmail: use the "threads: on" mode by default
with "threads: unread", there is a bit too much change between different times
of opening the mailbox. perhaps the mode wasn't a good idea...
2023-09-22 14:12:46 +02:00
89c543f662 if there is special-use junk flag on mailbox, don't also look at AutomaticJunkFlags option
the special-use flag should take precedence.
2023-09-22 10:51:42 +02:00
6315d57166 ignore new pprof files from test-upgrade.sh after previous commit 2023-09-21 18:42:01 +02:00
4de0af4fa5 add another automated upgrade test
for the path from v0.0.5 with lots of messages straight to the latest
development version. this can do multiple database changes in one go, so it's a
bit different than for installs where an admin has upgraded each version when
it was released.
2023-09-21 16:09:40 +02:00
d618cbf918 mention funding through nlnet/eu ngi0 entrust
small line in the readme, but this means a lot for the project: continued
development for a year. expect lots of improvements and features.
2023-09-21 16:08:43 +02:00
e6d8049548 webmail: in attachment viewer, for text/* content-type, show the text immediately too
instead of claiming it may be a binary file and showing a button to display the contents.
2023-09-21 15:29:38 +02:00
2e16d8025d when moving message to mailbox with special-use flag "Junk", mark the message as junk too, for retraining
i had been using the AutomaticJunkFlags option, so hadn't noticed the special use flag wasn't used.
2023-09-21 15:20:24 +02:00
79774c15ec add todo's about mime header parameter decoding
not sure what the correct approach is, would need to analyze email archive for practices.
2023-09-21 15:18:25 +02:00
f87f286b80 webmail: dragging works on selected items, so tell user they cannot drag if they try to drag a non-selected message 2023-09-21 14:39:40 +02:00
20f11409b6 webmail: when open the first unread message of a thread by default when opening a mailbox with threading enabled and the most recent message is in a thread 2023-09-21 12:56:51 +02:00
fc6e61e9a5 webmail: add arrow left/right to collapse/expanse threads 2023-09-21 11:51:38 +02:00
9bc860e207 webmail: make double click on mailbox expand/collapse, and make mailbox text unselectable (so the double click doesn't also select text) 2023-09-21 11:40:22 +02:00
941a2311f0 webmail: try a bit harder not to get mailbox names or search queries in the potential stacktrace
we want to user to submit the stack trace. user can still edit before
submitting, but it won't look attractive to submit stacktraces with info that
shouldn't be there. not great that firefox is including too much info and the
effort we need to make to get it out again, but well.
2023-09-21 11:31:07 +02:00
d07c871f5c webmail: better recognize URLs in text wrapped in () or <> if it follows interpunction
e.g. "text... (https://localhost)." would keep ) as part of the url before, but not anymore.
2023-09-21 11:09:27 +02:00
d649cf7dc2 quickstart: recognize likely NAT setup and set up host IPs in "NATIPs" field in the public listener
for issue #59 by pmarini, thanks!
2023-09-21 10:55:15 +02:00
cde54442d2 webmail: in status line about (re|dis)connecting, make error message more readable
with space after line, so a next line doesn't get concatenated. and with capital.
2023-09-21 09:07:49 +02:00
9534e464f9 add comment about the sconf config file format at the top of the config files
hopefully this helps admins editing the file and prevent mistakes about config files.

for issue #56 by kikoreis, thanks!
2023-09-21 08:59:10 +02:00
0d8603f9e1 update latest deps 2023-09-20 16:52:18 +02:00
ca5ef645f3 rename Account.Deliver to Account.DeliverDestination
the name was too generic compared with the other Deliver functions
2023-09-15 17:51:28 +02:00
3620d6f05e initialize metric mox_panic_total with 0, so the alerting rule also catches the first panic for a label
increase() and rate() don't seem to assume a previous value of 0 when a vector
gets a first value for a label. you would think that an increase() on a
first-value mox_panic_total{"..."}=1 would return 1, and similar for rate(), but
that doesn't appear to be the behaviour. so we just explicitly initialize the
count to 0 for each possible label value. mox has more vector metrics, but
panics feels like the most important, and it's too much code to initialize them
all, for all combinations of label values. there is probably a better way that
fixes this for all cases...
2023-09-15 16:47:17 +02:00
af71e9855b add package-level comments for webadmin and webaccount 2023-09-15 16:01:23 +02:00
bff0131164 webmail: new shortcut "T" for showing html version of email, and t for text version
shortcut X used to be "show html version", but with threading support became
"toggle collapse", so there was a clash.
2023-09-15 15:51:59 +02:00
3fb41ff073 implement message threading in backend and webmail
we match messages to their parents based on the "references" and "in-reply-to"
headers (requiring the same base subject), and in absense of those headers we
also by only base subject (against messages received max 4 weeks ago).

we store a threadid with messages. all messages in a thread have the same
threadid.  messages also have a "thread parent ids", which holds all id's of
parent messages up to the thread root.  then there is "thread missing link",
which is set when a referenced immediate parent wasn't found (but possibly
earlier ancestors can still be found and will be in thread parent ids".

threads can be muted: newly delivered messages are automatically marked as
read/seen.  threads can be marked as collapsed: if set, the webmail collapses
the thread to a single item in the basic threading view (default is to expand
threads).  the muted and collapsed fields are copied from their parent on
message delivery.

the threading is implemented in the webmail. the non-threading mode still works
as before. the new default threading mode "unread" automatically expands only
the threads with at least one unread (not seen) meessage. the basic threading
mode "on" expands all threads except when explicitly collapsed (as saved in the
thread collapsed field). new shortcuts for navigation/interaction threads have
been added, e.g. go to previous/next thread root, toggle collapse/expand of
thread (or double click), toggle mute of thread. some previous shortcuts have
changed, see the help for details.

the message threading are added with an explicit account upgrade step,
automatically started when an account is opened. the upgrade is done in the
background because it will take too long for large mailboxes to block account
operations. the upgrade takes two steps: 1. updating all message records in the
database to add a normalized message-id and thread base subject (with "re:",
"fwd:" and several other schemes stripped). 2. going through all messages in
the database again, reading the "references" and "in-reply-to" headers from
disk, and matching against their parents. this second step is also done at the
end of each import of mbox/maildir mailboxes. new deliveries are matched
immediately against other existing messages, currently no attempt is made to
rematch previously delivered messages (which could be useful for related
messages being delivered out of order).

the threading is not yet exposed over imap.
2023-09-13 15:44:57 +02:00
b754b5f9ac fix flushing of transparently compressed gzip output
this is a problem for connections like SSE, that only send data on events.
those events would stay in the gzip buffer until lots more data was written.

bug because of automatically typing "if err != nil"...

found while testing the maildir/mbox web-based import while working on message
threading support. the import gets progress SSE events that were now hanging.
2023-09-12 21:22:08 +02:00
6f1e38f2ce add flag to mox to store execution trace, similar to cpu/memory profiling
useful for performance testing
2023-09-12 14:43:52 +02:00
4a4ccb83a3 when making a message preview, also recognize []-enclosed "horizontal ellipsis" unicode character as a snip 2023-09-11 14:41:50 +02:00
fc7b0cc71e fix parsing List-Post header in webmail 2023-09-11 11:55:28 +02:00
f6d03a0eab track more unexpected panics in metrics 2023-09-11 11:43:49 +02:00
a5006a9090 fix not adding duplicate domains to the list of "verified dkim domains" for incoming messages 2023-09-11 11:37:45 +02:00
cb1b133e28 add more rfc's, for jmap, caldav, carddav, lemonade profile
being on the list does not mean it is implemented.
2023-09-11 11:26:40 +02:00
a6ae87d7ac webmail: fix showing attachments that are text/plain and have content-disposition: attachment
they were not added to the list of attachments when sending the message to the
webmail frontend. they were shown on the "open message in new tab" page.
2023-09-03 15:20:56 +02:00
4283ceecfc fix serving static webmail files in development mode
due to a missing return, the content was served again.
this path doesn't happen on release binaries, only during local development,
where there is a local file that can be served.
2023-09-03 15:17:09 +02:00
165639cb38 smtpserver: in helo/ehlo for submission don't fail on bad domain/ip address
for submission, we don't care about the value. users typically won't be able to
easily fix the errors (of their mail client software). so we just ignore the
domain/ip address, unless in pedantic mode.

this also parses "additional information after literal addresses" more
strictly/correctly.

for issue #55 by gimpf, thanks for the report!
2023-08-25 15:29:02 +02:00
f4c20673ff don't generate duplicate spf record if hostname is equal to domain name, e.g. postmaster@mail.domain.
the assumption has been that the hostname is something like mail.<domain>, when
setting up mox with the quickstart for user@<domain>. but users can use the
quickstart for postmaster@mail.<domain> as well.

for issue #46 by x8x, thanks for reporting!
2023-08-25 14:32:28 +02:00
61a5eb61a4 remove needless fmt.Sprintf
by staticcheck
2023-08-23 16:27:02 +02:00
f029db3f47 imapserver bugfix: fix expunging for messages marked junk/nonjunk
such messages would be marked expunged in the database, then the junkfilter
would be retrained for the removal of the message. but during retraining, the
expunged flag would be cleared again. the on-disk message file would still be
removed. so when opening the mailbox, the message would appear to still exist,
but cannot be retrieved from disk.

if you run "mox fixmsgsize", and you get warnings about missing message files,
you could create empty files (with "touch"), run "mox fixsmsgsize" again,
followed by "mox recalculatemailboxcounts <affectedaccount>" and run "mox
bumpuidvalidity <affectaccount>".

"mox backup" would probably also complain, as would "mox verifydata".

this may have caused the "wrong mailbox counts" error i got from "mox
verifydata" on a backup.
2023-08-23 16:20:06 +02:00
da9f1d9d0d in admin pages, make the literal instruction text on the dnscheck page visible, and set a max-width for easier readability 2023-08-23 15:10:02 +02:00
b3dd4a55c3 fix a spello, and reword so misspell doesn't complain about it 2023-08-23 14:59:43 +02:00
affb057a0c webmail: fix case where tree of mailboxes wasn't displayed properly
for example, when these mailboxes existed: "a", "a.b", "a/b", then "a.b" (.
before / in ascii) prevented "a/b" from being displayed in the tree below "a".
2023-08-23 14:57:05 +02:00
aebfd78a9f implement accepting dmarc & tls reports for other domains
to accept reports for another domain, first add that domain to the config,
leaving all options empty except DMARC/TLSRPT in which you configure a Domain.

the suggested DNS DMARC/TLSRPT records will show the email address with
configured domain. for DMARC, the dnscheck functionality will verify that the
destination domain has opted in to receiving reports.

there is a new command-line subcommand "mox dmarc checkreportaddrs" that
verifies if dmarc reporting destination addresses have opted in to received
reports.

this also changes the suggested dns records (in quickstart, and through admin
pages and cli subcommand) to take into account whether DMARC and TLSRPT is
configured, and with which localpart/domain (previously it always printed
records as if reporting was enabled for the domain). and when generating the
suggested DNS records, the dmarc.Record and tlsrpt.Record code is used, with
proper uri-escaping.
2023-08-23 14:27:21 +02:00
9e248860ee implement transparent gzip compression in the webserver
we only compress if applicable (content-type indicates likely compressible),
client supports it, response doesn't already have a content-encoding).

for internal handlers, we always enable compression.  for reverse proxied and
static files, compression must be enabled per handler.

for internal & reverse proxy handlers, we do streaming compression at
"bestspeed" quality (probably level 1).

for static files, we have a cache based on mtime with fixed max size, where we
evict based on least recently used. we compress with the default level (more
cpu, better ratio).
2023-08-21 21:52:35 +02:00
4c72184b44 update link to docker image
user was being redirected to the new url
2023-08-20 18:45:19 +02:00
b43529a2e9 sendmail: bugfix: set remote hostname to verify for tls connections
due to logic bug we weren't setting it, and tls connections would fail with a
warning that either the remote hostname must be set or insecurityskipverify
must be set.
2023-08-20 18:26:20 +02:00
0b9475271c add possible future todo for working around ios messages with wrong q-encoded headers v0.0.6 2023-08-16 16:22:00 +02:00
80547df6ee webmail: don't have two spaces between header and address(es) (e.g. for From/To)
because outlook.com will reformat the message and then fail to verify the message.
proton.me also reformats and invalidates the dkim signature, but seemingly
after it verifies the dkim signature.
2023-08-16 15:22:38 +02:00
1ccc5d0177 fix message size in a message in gentestdata
and work around the message in test-upgrade.sh.
and add subcommand to open an account, triggering data upgrades.
2023-08-16 14:36:17 +02:00
ddf3cb3653 mention there are now webmail screenshots, and small release process tweaks 2023-08-16 10:16:48 +02:00
9f46879377 webmail: correct label for Subject in search form 2023-08-15 13:03:02 +02:00
aed23d900a update dependencies 2023-08-15 10:58:01 +02:00