Commit Graph

946 Commits

Author SHA1 Message Date
6f1e38f2ce add flag to mox to store execution trace, similar to cpu/memory profiling
useful for performance testing
2023-09-12 14:43:52 +02:00
4a4ccb83a3 when making a message preview, also recognize []-enclosed "horizontal ellipsis" unicode character as a snip 2023-09-11 14:41:50 +02:00
fc7b0cc71e fix parsing List-Post header in webmail 2023-09-11 11:55:28 +02:00
f6d03a0eab track more unexpected panics in metrics 2023-09-11 11:43:49 +02:00
a5006a9090 fix not adding duplicate domains to the list of "verified dkim domains" for incoming messages 2023-09-11 11:37:45 +02:00
cb1b133e28 add more rfc's, for jmap, caldav, carddav, lemonade profile
being on the list does not mean it is implemented.
2023-09-11 11:26:40 +02:00
a6ae87d7ac webmail: fix showing attachments that are text/plain and have content-disposition: attachment
they were not added to the list of attachments when sending the message to the
webmail frontend. they were shown on the "open message in new tab" page.
2023-09-03 15:20:56 +02:00
4283ceecfc fix serving static webmail files in development mode
due to a missing return, the content was served again.
this path doesn't happen on release binaries, only during local development,
where there is a local file that can be served.
2023-09-03 15:17:09 +02:00
165639cb38 smtpserver: in helo/ehlo for submission don't fail on bad domain/ip address
for submission, we don't care about the value. users typically won't be able to
easily fix the errors (of their mail client software). so we just ignore the
domain/ip address, unless in pedantic mode.

this also parses "additional information after literal addresses" more
strictly/correctly.

for issue #55 by gimpf, thanks for the report!
2023-08-25 15:29:02 +02:00
f4c20673ff don't generate duplicate spf record if hostname is equal to domain name, e.g. postmaster@mail.domain.
the assumption has been that the hostname is something like mail.<domain>, when
setting up mox with the quickstart for user@<domain>. but users can use the
quickstart for postmaster@mail.<domain> as well.

for issue #46 by x8x, thanks for reporting!
2023-08-25 14:32:28 +02:00
61a5eb61a4 remove needless fmt.Sprintf
by staticcheck
2023-08-23 16:27:02 +02:00
f029db3f47 imapserver bugfix: fix expunging for messages marked junk/nonjunk
such messages would be marked expunged in the database, then the junkfilter
would be retrained for the removal of the message. but during retraining, the
expunged flag would be cleared again. the on-disk message file would still be
removed. so when opening the mailbox, the message would appear to still exist,
but cannot be retrieved from disk.

if you run "mox fixmsgsize", and you get warnings about missing message files,
you could create empty files (with "touch"), run "mox fixsmsgsize" again,
followed by "mox recalculatemailboxcounts <affectedaccount>" and run "mox
bumpuidvalidity <affectaccount>".

"mox backup" would probably also complain, as would "mox verifydata".

this may have caused the "wrong mailbox counts" error i got from "mox
verifydata" on a backup.
2023-08-23 16:20:06 +02:00
da9f1d9d0d in admin pages, make the literal instruction text on the dnscheck page visible, and set a max-width for easier readability 2023-08-23 15:10:02 +02:00
b3dd4a55c3 fix a spello, and reword so misspell doesn't complain about it 2023-08-23 14:59:43 +02:00
affb057a0c webmail: fix case where tree of mailboxes wasn't displayed properly
for example, when these mailboxes existed: "a", "a.b", "a/b", then "a.b" (.
before / in ascii) prevented "a/b" from being displayed in the tree below "a".
2023-08-23 14:57:05 +02:00
aebfd78a9f implement accepting dmarc & tls reports for other domains
to accept reports for another domain, first add that domain to the config,
leaving all options empty except DMARC/TLSRPT in which you configure a Domain.

the suggested DNS DMARC/TLSRPT records will show the email address with
configured domain. for DMARC, the dnscheck functionality will verify that the
destination domain has opted in to receiving reports.

there is a new command-line subcommand "mox dmarc checkreportaddrs" that
verifies if dmarc reporting destination addresses have opted in to received
reports.

this also changes the suggested dns records (in quickstart, and through admin
pages and cli subcommand) to take into account whether DMARC and TLSRPT is
configured, and with which localpart/domain (previously it always printed
records as if reporting was enabled for the domain). and when generating the
suggested DNS records, the dmarc.Record and tlsrpt.Record code is used, with
proper uri-escaping.
2023-08-23 14:27:21 +02:00
9e248860ee implement transparent gzip compression in the webserver
we only compress if applicable (content-type indicates likely compressible),
client supports it, response doesn't already have a content-encoding).

for internal handlers, we always enable compression.  for reverse proxied and
static files, compression must be enabled per handler.

for internal & reverse proxy handlers, we do streaming compression at
"bestspeed" quality (probably level 1).

for static files, we have a cache based on mtime with fixed max size, where we
evict based on least recently used. we compress with the default level (more
cpu, better ratio).
2023-08-21 21:52:35 +02:00
4c72184b44 update link to docker image
user was being redirected to the new url
2023-08-20 18:45:19 +02:00
b43529a2e9 sendmail: bugfix: set remote hostname to verify for tls connections
due to logic bug we weren't setting it, and tls connections would fail with a
warning that either the remote hostname must be set or insecurityskipverify
must be set.
2023-08-20 18:26:20 +02:00
0b9475271c add possible future todo for working around ios messages with wrong q-encoded headers v0.0.6 2023-08-16 16:22:00 +02:00
80547df6ee webmail: don't have two spaces between header and address(es) (e.g. for From/To)
because outlook.com will reformat the message and then fail to verify the message.
proton.me also reformats and invalidates the dkim signature, but seemingly
after it verifies the dkim signature.
2023-08-16 15:22:38 +02:00
1ccc5d0177 fix message size in a message in gentestdata
and work around the message in test-upgrade.sh.
and add subcommand to open an account, triggering data upgrades.
2023-08-16 14:36:17 +02:00
ddf3cb3653 mention there are now webmail screenshots, and small release process tweaks 2023-08-16 10:16:48 +02:00
9f46879377 webmail: correct label for Subject in search form 2023-08-15 13:03:02 +02:00
aed23d900a update dependencies 2023-08-15 10:58:01 +02:00
02a03710dc don't try to (non-recursively) remove directories from the data tmp dir
mox only creates files there. directories could be a backup that is being
transferred to elsewhere.
2023-08-15 09:51:52 +02:00
fdbbfb765b point users to spamhaus and spamcop pages and terms of use 2023-08-15 09:48:53 +02:00
983002b074 with strict message parsing, don't allow lines longer than 1000 bytes 2023-08-15 09:21:36 +02:00
34c2dcd49d add strict mode when parsing messages, typically enabled for incoming special-use messages like tls/dmarc reports, subjectpass emails
and pass a logger to the message parser, so problems with message parsing get
the cid logged.
2023-08-15 08:25:56 +02:00
f5f953b3ab handle parsing message header without header/body separator
the commit before the previous added tests with a message with only 1 header
line. it's a valid message, but Go's mail.ReadMessage doesn't handle it with
go1.20 and earlier. the automated "test with previous go release" caught it.
work around it by adding the expected but absent \r\n to the parse function.
2023-08-14 15:40:27 +02:00
f96310fdd5 fix checking for tls certificates, and the quickstart with the -existing-webserver flag
some time ago, the flag to ParseConfig() to do or skip checking the tls
keys/certs was inverted, but it looks like i didn't change the call sites... so
during "mox config test", and after a regular "mox quickstart" there was no
check for the tls keys/certs, and during "mox quickstart -existing-webserver"
there was a check where there shouldn't be. this made using -existing-webserver
impossible.

this became clear with the question by morki in issue #5.
2023-08-14 15:01:17 +02:00
48eb530b1f improve message parsing: allow bare carriage return (unless in pedantic mode), allow empty header, and no longer treat a message with only headers as a message with only a body 2023-08-11 14:07:49 +02:00
79d06184ab fix flaky test, event doesn't have to be set 2023-08-11 10:46:22 +02:00
55d05c6bea replace listener config option IPsNATed with NATIPs, and let autotls check NATIPs
NATIPs lists the public IPs, so we can still do the DNS checks on them. with
IPsNATed, we disabled the checks.

based on feedback by kikoreis in issue #52
2023-08-11 10:13:17 +02:00
d7df70acd8 webmail: don't lose display of additional headers when a flag/keyword changes (e.g. marked as read) 2023-08-11 08:38:57 +02:00
383eb483df webmail: for html-only messages, also show the "show html with external resources" button 2023-08-10 14:55:30 +02:00
a4c6fe815f make some maintenance commands that were previously unlisted listed
we refer to these commands in output of "mox verifydata", so they should be
findable other than through the code...
2023-08-10 12:29:46 +02:00
7cceb3d834 add comment about not verifying Sender for submissions 2023-08-10 12:18:05 +02:00
6b68920a3a Go's LookupAddr will return non-absolute names, seemingly for single-label names from /etc/hosts, turn them into absolute names so our verifying forward lookups can succeed 2023-08-10 11:52:35 +02:00
a30d8c1378 for localserve, don't special-case smtp submit
the recent webmail addition added localserve local delivery in queue.Add, so we
just that for smtpserver too.

and don't drop incoming smtp deliver messages, but deliver as normal.
2023-08-10 11:28:57 +02:00
ce91b7d23e update roadmap 2023-08-10 11:05:38 +02:00
0434e49c3a webmail: while attachment viewer is open, don't handle global keyboard shortcuts (like search, going to inbox)
feedback from jonathan, thanks!
2023-08-10 11:02:13 +02:00
c24bb063e5 webmail tweaks
- padding on small attachment download button.
- don't remember "show html" but always display text first.
- propagate modseq to message when flags/keywords change, so "show internals" shows the update.
2023-08-10 10:56:04 +02:00
f48a53726e when clearing search, open inbox
feedback from jonathan, thanks!
2023-08-10 10:42:54 +02:00
038b478d16 listen/bind in deterministic order for consistent error messages, and warn if quickstart cannot find public ip's
without public ip's, the generated mox config will try to listen on 0.0.0.0 and
::, but because there is already a listener for 127.0.0.1:80 (and possibly
others), a bind for 0.0.0.0:80 will fail. explicit public ip's are needed.

the public http listener is useful for ACME validation over http.

for issue #52
2023-08-10 10:29:06 +02:00
01bcd98a42 add flag to ruleset that indicates a message is forwarded, slightly modifying how junk analysis is done
part of PR #50 by bobobo1618
2023-08-09 22:31:37 +02:00
9c31789c56 add option to ruleset to accept incoming spammy messages to a configured mailbox
this is based on @bobobo1618's PR #50. bobobo1618 had the right idea, i tried
including an "is forwarded email" configuration option but that indeed became
too tightly coupled. the "is forwarded" option is still planned, but it is
separate from the "accept rejects to mailbox" config option, because one could
still want to push back on forwarded spam messages.

we do an actual accept, delivering to a configured mailbox, instead of storing
to the rejects mailbox where messages can automatically be removed from.  one
of the goals of mox is not pretend to accept email while actually junking it.
users can still configure delivery to a junk folder (as was already possible),
but aren't deleted automatically. there is still an X-Mox-Reason header in the
message, and a log line about accepting the reject, but otherwise it is
registered and treated as an (smtp) accept.

the ruleset mailbox is still required to keep that explicit. users can specify
Inbox again.

hope this is good enough for PR #50, otherwise we'll change it.
2023-08-09 22:25:10 +02:00
383fe4f53a explicitly store in a Message whether it was delivered to the rejects mailbox
soon, we can have multiple rejects mailboxes.  and checking against the
configured rejects mailbox name wasn't foolproof to begin with, because it may
have changed between delivery to the rejects mailbox and the message being
moved.

after upgrading, messages currently in rejects mailboxes don't have IsReject
set, so they don't get the special rejecs treatment when being moved. they are
removed from the rejects mailbox after some time though, and newly added
rejects will be treated correctly. so this means some existing messages wrongly
delivered to the rejects mailbox, and moved out, aren't used (for a positive
signal) for future deliveries.  saves a bit of complexity in the
implementation.  i think the tradeoff is worth it.

related to discussion in issue #50
2023-08-09 16:52:24 +02:00
0fc59af9a8 add Deliver-To header for delivered messages
for (experimental) rfc 9228
2023-08-09 10:20:45 +02:00
20ebdae8ea in webmail, automatically mark message as nonjunk when open for 5 seconds, and prevent extraneous newlines when composing a reply to selected text 2023-08-09 09:45:54 +02:00