python - UnboundLocalError running .py from .bat (with API); Runs fine in Command Prompt

Question

When I execute my commands in the Command Prompt on Windows 10 Python 3.8.2, I get the expected behavior and my data saves as it should (1.0.3 printed for troubleshooting):

    Microsoft Windows [Version 10.0.19041.746]
(c) 2020 Microsoft Corporation. All rights reserved.

C:Users>SET BEARER_TOKEN=xyz

C:Users>cd C:UsersTwitterConv

C:UsersTwitterConv>python scrape-test.py neonphotography
1.0.3

C:UsersTwitterConv>

However when I put this into a .bat file I get the following error when running it from anywhere:

C:UsersTwitterConv>python scrape-test.py neonphotography
1.0.3
Traceback (most recent call last):
      File "scrape-test.py", line 333, in <module>
        main()
      File "scrape-test.py", line 322, in main
        user_ids = get_user_info(headers, filename)
      File "scrape-test.py", line 155, in get_user_info
        user_df = pandas.json_normalize(json_response["data"])
    UnboundLocalError: local variable 'json_response' referenced before assignment

Here is the .bat file:

SET BEARER_TOKEN=xyz
cd /D C:UsersTwitterConv
python scrape-test.py neonphotography

I'm trying to adapt this project from coworkers using Mac's and I really expected this to be easy once I started reading about batch files. I've tried using pyinstaller as a work around but that's a whole new set of issues, so I figured I would start here.

Why is my batch file not running properly?

question from:https://stackoverflow.com/questions/65893702/unboundlocalerror-running-py-from-bat-with-api-runs-fine-in-command-prompt

Answer 1

It's useful to look at the stack trace itself. The only way your script can fail like that is if the call to twitter is failing. In this case, the only thing that could reasonably differ with the batch file versus the interactive command line is somehow the bearer token is different. Since bearer tokens often have percent symbols in them, there's probably something going on there.

We can simplify this with a simple script:

# This is example.py
import os
print("BEARER_TOKEN=[" + os.environ.get("BEARER_TOKEN") + "]")

If you run this interactively, it works as you'd expect:

C:Example>set BEARER_TOKEN=this%is%an%example%token

C:Example>python example.py
BEARER_TOKEN=[this%is%an%example%token]

However, running it with a simple batch file, it will not work as expected:

@echo off
rem This is example.cmd
set BEARER_TOKEN=this%is%an%example%token
python3 example.py

C:Example>example.cmd
BEARER_TOKEN=[thisantoken]

You'll need to escape the percent symbols in your batch file, since they're parsed differently than the interactive prompt:

@echo off
rem This is the fixed example.cmd
set BEARER_TOKEN=this%%is%%an%%example%%token
python3 example.py

C:Example>example.cmd
BEARER_TOKEN=[this%is%an%example%token]