after seeing some junk messages pass the filter, i investigated word counts in
junkfilter.db. i had seen suspicious counts that were just around powers of
two. did not make sense at the time. more investigating makes it clear: instead
of setting new word counts when updating the junk filter, we were adding the
new value to the current value (instead of just setting the new value). so the
counts got approximately doubled when being updated.
users should retrain the junk filter after this update using the "retrain"
subcommand.
this also adds logging for the hypothetical case where numbers would get
decreased below zero (which would wrap around due to uints).
and this fixes junk filter tests that were passing wrong parameters to
train/untrain...
the types in webhook should be subjected to apidiff'ing, this was a shared
function. it is better off in package queue. also change the apidiff script so
it leaves apidiff/next.txt empty when there aren't any changes. makes it easier
to rotate the files after releases where nothing changed (a common occurrence).
these settings are applied anywhere the webmail is open. the settings are for
showing keyboard shortcuts in the lower right after a mouse interaction, and
showing additional headers. the shorcuts were configurable in the "help" popup
before. the additional headers were only configurable through the developer
console before.
the "mailto:" (un)register buttons are now in the settings popup too.
ctrl-l is commonly "focus on browser address bar".
ctrl-u is commonly "view source".
ctrl-I (shift i) is commonly "open developer console".
these keys are more useful to leave for the browser. ctrl-l and ctrl-u (moving
to a message without opening it) can still be had by using also pressing shift.
the previous ctrl-shift-i (show all headers) is now just ctrl-i.
this has been requested in the past on irc/matrix (i forgot who).
the field is optional. if absent, the default behaviour is currently to disable
session tickets. users can set the option if they want to try if delivery from
microsoft is working again. in a future version, we can switch the default to
enabling session tickets.
the previous fix was to disable session tickets for all tls connections,
including https. that was a bit much.
for issue #237
for each message part. The ContentDisposition value is the base value without
header key/value parameters. the Filename field is the likely filename of the
part. the different email clients encode filenames differently. there is a
standard mime mechanism from rfc 2231. and there is the q/b-word-encoding from
rfc 2047. instead of letting users of the webhook api deal with those
differences, we provide just the parsed filename.
for issue #258 by morki, thanks for reporting!
the imap & smtp servers now allow logging in with tls client authentication and
the "external" sasl authentication mechanism. email clients like thunderbird,
fairemail, k9, macos mail implement it. this seems to be the most secure among
the authentication mechanism commonly implemented by clients. a useful property
is that an account can have a separate tls public key for each device/email
client. with tls client cert auth, authentication is also bound to the tls
connection. a mitm cannot pass the credentials on to another tls connection,
similar to scram-*-plus. though part of scram-*-plus is that clients verify
that the server knows the client credentials.
for tls client auth with imap, we send a "preauth" untagged message by default.
that puts the connection in authenticated state. given the imap connection
state machine, further authentication commands are not allowed. some clients
don't recognize the preauth message, and try to authenticate anyway, which
fails. a tls public key has a config option to disable preauth, keeping new
connections in unauthenticated state, to work with such email clients.
for smtp (submission), we don't require an explicit auth command.
both for imap and smtp, we allow a client to authenticate with another
mechanism than "external". in that case, credentials are verified, and have to
be for the same account as the tls client auth, but the adress can be another
one than the login address configured with the tls public key.
only the public key is used to identify the account that is authenticating. we
ignore the rest of the certificate. expiration dates, names, constraints, etc
are not verified. no certificate authorities are involved.
users can upload their own (minimal) certificate. the account web interface
shows openssl commands you can run to generate a private key, minimal cert, and
a p12 file (the format that email clients seem to like...) containing both
private key and certificate.
the imapclient & smtpclient packages can now also use tls client auth. and so
does "mox sendmail", either with a pem file with private key and certificate,
or with just an ed25519 private key.
there are new subcommands "mox config tlspubkey ..." for
adding/removing/listing tls public keys from the cli, by the admin.
needed for upcoming changes, where (now) package admin needs to import package
store. before, because package store imports mox- (for accessing the active
config), that would lead to a cyclic import. package mox- keeps its active
config, package admin has the higher-level config-changing functions.
this allows removing some ugly instantiations of an rng based on the current
time.
Intn is now IntN for our concurrency-safe prng wrapper to match the randv2 api.
v2 exists since go1.22, which we already require.
and add an alerting rule if the failure rate becomes >10% (e.g. expired
certificate).
the prometheus metrics includes a reason, including potential tls alerts, if
remote smtp clients would send those (openssl s_client -starttls does).
inspired by issue #237, where incoming connections were aborted by remote. such
errors would show up as "eof" in the metrics.
if files {webmail,webaccount,webadmin}.{css,js} exist in the configdir (where
the mox.conf file lives), their contents are included in the web apps.
the webmail now uses css variables, mostly for colors. so you can write a
custom webmail.css that changes the variables, e.g.:
:root {
--color: blue
}
you can also look at css class names and override their styles.
in the future, we may want to make some css variables configurable in the
per-user settings in the webmail. should reduce the number of variables first.
any custom javascript is loaded first. if it defines a global function
"moxBeforeDisplay", that is called each time a page loads (after
authentication) with the DOM element of the page content as parameter. the
webmail is a single persistent page. this can be used to make some changes to
the DOM, e.g. inserting some elements. we'll have to see how well this works in
practice. perhaps some patterns emerge (e.g. adding a logo), and we can make
those use-cases easier to achieve.
helps partially with issue #114, and based on questions from laura-lilly on
matrix.
the pointer events for moving the mouse would be consumed by the iframe. that
broke resizing of the compose popup. we now disable pointerevents on the main
ui when we are dragging the corner of the compose popup.
this is similar to an earlier change about the draggable split bar between the
message list and the message view (when showing an html message).
we kept the "save draft" promise, and would wait for it again for other
operations (eg close, save again, send), which wouldn't make progress.
can easily be reproduced by saving a message with a control character in an
address or the subject. saving the draft will fail.
for issue #256 by ally9335, thanks for reporting
since ios18, downloaded files don't go immediately to the settings (which is
somewhat understandable given potential for abuse), but go to the Files app.
opening them in the Files app then adds them to the settings where they can be
installed.
if admin has disabled mta-sts for a domain, we still check for records &
policies, but won't mark it as error when they don't exist. we do now keep
warning that mta-sts isn't enabled, otherwise we would start showing a green
"ok".
this also fixes the mta-sts code returning ErrNoPolicy when mtasts.<domain>
doesn't exist. the dns lookup is done with the reguler "net" package dns lookup
code, not through adns, so we look for two types of DNSError's.
noticed a while ago when testing with MTA-STS while debugging TLS connection
issues with MS.
was encountered during smtp session. but could also happen for imapserver and
webmail.
in smtpserver, we now log error messages for smtp errors that cause us to print
a stack trace. would have made logging output more helpful (without having to
turn on trace-level logging).
hopefully solves issue #238 by mwyvr, thanks for reporting!
and explain in more detail what it means in the webadmin interface.
will hopefully bring less confusion.
for issue #244 by exander77, thanks for reporting
"bodystructure" is like "body", but bodystructure allows returning more
information. we chose not to do that, initially because it was easier to
implement, and more recently because we can't easily return the additional
content-md5 field for leaf parts (since we don't have it in parsed form). but
now we just return the extended form for multiparts, and non-extended form for
leaf parts. likely no one would be looking for any content-md5-value for leaf
parts anyway. knowing the boundary is much more likely to be useful.
for issue #217 by danieleggert, thanks for reporting!
before this change, we were logging an empty string, which turned into "[]",
looking like an empty array. misleading and unhelpful.
this is fixed by making struct fields on type recipient "exported" so they can
get logged, and by changing the logging code to log nested
struct/pointer/interface fields if we would otherwise wouldn't log anything
(when only logging more basic data types).
we'll now get log lines like:
l=info m="deliver attempt to unknown user(s)" pkg=smtpserver recipients="[addr=bogus@test.example]"
for issue #232 by snabb, thanks for reporting!
before this change, when archiving, we would move all messages from the thread
that are in the same mailbox as that of the response message to the archive
mailbox. so if the message that was being responsed to was already in the
archive mailbox, the message would be moved from archive mailbox to archive
mailbox, resulting in an error.
with this change, when archiving, we move the thread messages that are in the
same mailbox as is currently open (independent of the mailbox the message lives
in, a common situation in the threading view). if there is no open mailbox
(search results), we still use the mailbox of the message being responded to as
reference.
with this new approach, we won't get errors moving a message to an archive
mailbox when it's already there. well, you can still get that error, but then
you've got the archive mailbox open, or you're in a search result and
responding to an archived message. the error should at least help understand
that nothing is happening.
we are only moving the messages from one active/reference mailbox because we
don't want to move messages from the thread that are in the Sent mailbox, and
we also don't want to move duplicate messages (cross-posts to mailing lists)
that are in other mailboxes. moving only the messages from the current active
mailbox seems safe, and should do what is what users would expect most of the
time.
for issue #233 by mattfbacon, thanks for reporting!
because this mechanism is most needed behind a reverse proxy, where acme
tls-alpn-01 won't work (because the reverse proxy won't pass on the alpn
extensions). if that's the case, there is obviously a webserver on port 443.
and it likely also running on port 80. so before this change, if tls-alpn-01
isn't available, http-01 also wasn't available, leaving no validation
mechanisms.
for issue #218 by mgkirs, thanks for reporting and details. hope this helps.
when we send a list of messages from the mox backend to the js frontend, we
include a parsed form of the "initial" message: the one we immediately show,
typically the top-most (unread) message. however, if that message could not be
parsed (due to invalid header syntax), we would fail the entire operation of
loading the view.
with this change, we simply don't return a parsed form of an initial message if
we cannot parse it. that will cause the webmail frontend to not select &
display a message immediately. if you then try to open the message, you'll
still get an error message as before. but at least the view has been loaded,
and you can open the raw message to inspect the contents.
for issue #219 by wneessen
not generating it yet from imapserver because we don't have content-md5
available. we could send "nil" instead of any actual content-md5 header (and
probably no contemporary messages include a content-md5 header), but it would
not be correct. if no known clients have problems in practice with absent
extensible data, it's better to just leave the bodystructure as is, with
extensible data.
for issue #217 by danieleggert
we add various information while analysing an incoming message. like
dkim/spf/ip reputation. and content-based junk filter threshold/result and
ham/spam words used.
for issue #179 by Fell and #157 by mattfbacon
