Python Unicode troubles - How can I use a text file as an e-mail body?

Question

What a surprise another person with a Unicode problem.

My text gets copied to my e-mail, but only after having been encoded to utf-8 countless times - and even then it is mixed with 50/50 gibberish.

In the past I have avoided this nuisance by using pathlibs encoding, but that doesn't appear to be possible here. I think this is the most important code for you guys.

#Write info to text file
with io.open(f'today.txt', "a", encoding = 'utf-8') as wf:
        wf.write(str(today) + '
')
        wf.write(item + '
')
        wf.write(text + '

')
        wf.write(urlbuff + href + '


')

#Store text for e-mail use
with io.open(f'today.txt', "r+", encoding = 'utf-8') as nwf:
        text_to_mail = str(nwf.readlines())
        text_mail = text_to_mail.encode('utf-8', 'strict')

My text_mail output looks like this:

b'['2021-01-22 ', 'Wealth tax ', "A wealth tax isn't perfect but it's not Armageddon ", ' '

If I don't use all of the separate encoding commands I get ascii errors and no output. How can I fix this problem, and ensure it never happens to me again? I'm sure there must be a far cleaner, more sensible workaround than what I've been trying. I'm using smtplib to send the e-mail.

I'm aware there's a lot of code you can't see, but I hope there's enough here nonetheless.

question from:https://stackoverflow.com/questions/65846658/python-unicode-troubles-how-can-i-use-a-text-file-as-an-e-mail-body

Answer 1

Thanks for the comments, I changed the with statement to this:

with io.open(f'today.txt', "r") as nwf:
    text_mail = nwf.read()

I also had to change the smtplib file in my pc's AppData directory, so that a few of the ascii .encode() statements used .encode('utf-8') instead. The changes I made were as follows:

self.putcmd("data")
    (code, repl) = self.getreply()
    if self.debuglevel > 0:
        self._print_debug('data:', (code, repl))
    if code != 354:
        raise SMTPDataError(code, repl)
    else:
        if isinstance(msg, str):
            msg = _fix_eols(msg).encode('ascii')

to:

self.putcmd("data")
    (code, repl) = self.getreply()
    if self.debuglevel > 0:
        self._print_debug('data:', (code, repl))
    if code != 354:
        raise SMTPDataError(code, repl)
    else:
        if isinstance(msg, str):
            msg = _fix_eols(msg).encode('utf-8')

If anyone needs to do this for themselves search for 'ascii', change the "msg = _fix_eols(msg).encode('ascii')" statements from above, I believe there are two. I'm not entirely aware of all possible implications for the aforementioned changes, so another fix would likely be better.