Get the last page Number of a wabpage - Beautiful Soup

I'm trying to get the page number of the last page of this website http://digitalmoneytimes.com/category/crypto-news/

This links shows that the last page number is 335 but i can't extract the page number.

soup = BeautifulSoup(page.content, 'html.parser')
soup_output= soup.find_all("li",{"class":"active"})
soup_output=soup.select(tag)
print(soup_output)

I get an empty list as the output

csspython-3.xweb-scrapingbeautifulsoup

Answers

answered 3 months ago Fozoro #1

In order to get the last page of the given website, I would strongly recommend you to use the following code:

import requests 
from bs4 import BeautifulSoup

page = requests.get("http://digitalmoneytimes.com/category/crypto-news/")
soup = BeautifulSoup(page.content, 'html.parser')
soup = soup.find_all("a", href = True)
pages = []
for x in soup:
    if "http://digitalmoneytimes.com/category/crypto-news/page/" in str(x):
        pages.append(x)
last_page = pages[2].getText()

where last_page is equal to the last page. Due to the fact that I don't have access to your tag and page variables I can't really tell you where is the problem in your code.

Really hope this solves your problem.

answered 3 months ago SIM #2

If it is about getting the last page number, there is something you might try out as well:

import requests
from bs4 import BeautifulSoup

link = 'http://digitalmoneytimes.com/category/crypto-news/'

res = requests.get(url)
soup = BeautifulSoup(res.text,"lxml")
last_page_num = soup.find(class_="pagination-next").find_previous_sibling().text
print(last_page_num)

Output:

336

comments powered by Disqus