Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
314 views
in Technique[技术] by (71.8m points)

python - How to write links to a text file?

I'm testing the following script:

import re
import requests
from bs4 import BeautifulSoup
import os
import fileinput

Link = 'https://en.wikipedia.org/wiki/Category:1990'
q = requests.get(Link)
soup = BeautifulSoup(q.text)
#print soup
subtitles = soup.findAll('div',{'class':'links'})
#print subtitles


with  open("Anilinks.txt", "w") as f:
    for link in subtitles:
        x = link.find_all('a', limit=26)
        for a in x:
            url = a['href']
            f.write(url+'
')

I'm trying to get each link copied/pasted to a text file. The script seems like it should work, but it doesn't actually do anything.

How can I get this working?

See Question&Answers more detail:os

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)

You can write a string to a text file like this:

with open("yourfile.txt", "w") as f: f.write(yourstr)

If you don't want to override the file just use "a" as second parameter in open. See https://docs.python.org/2/library/functions.html#open.

So, I assume that you have a list of links like this:

["http://example.com", "http://stackoverflow.com"]

and you want to have a file like this:

http://example.com:
<!doctype html>
<html>
<body>
...
<h1>Example Domain</h1>
...
</body>
</html>

http://stackoverflow.com:
...

Let's start with iterating all the links:

for url in yourlinks:

First, you want to write the url to the file:

    with open("yourfile.txt", "a") as f:
        f.write(url+"
") # the 
 is a new line

Now you download the content of the website to a variable:

        content = urllib2.urlopen(url).read()

(It may be that there are errors because of encoding - I'm from python3.) And you write it to the file:

        f.write(content+"
")

Voila! You should have your file now.


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...