How to use Python Scrapy if CSS or XPATH selectors don't work?

I'm trying to scrape from this site using scrapy shell.

    https://api.simon.com/v1.2/tenant?mallId=1345&key=40A6F8C3-3678-410D-86A5-BAEE2804C8F2&lw=true

My goal is to extract all the tenant information.

However, I'm having difficulties as it seems that both my CSS and XML selectors aren't working, or at least they're not returning any information. The website says that it's an XML file without any style information associated with it. I'm not sure what that means.

I've narrowed down the problem and am now just trying to return information on a single tenant/store but I still can't seem to figure it out. Here are some of the selectors that I've tried

    response.xpath('//*[@id="collapsible2"]/div[1]/div[1]/span[2]/text()').extract() 
    response.css("#collapsible2 > div.expanded > div:nth-child(1) > span.html-tag::text").extract()

Thanks in advance

pythoncssxpathweb-scrapingscrapy

Answers

comments powered by Disqus