Extract all links from web pages

To extract all links from web pages, you generally need to:

1. Fetch the web page’s HTML content.

2. Parse the HTML.

3. Extract all <a> tags with their href attributes.

Here are some common methods for extracting links:

1. Using Python (with BeautifulSoup)

python
CopyEdit
import requests
from bs4 import BeautifulSoup

url = 'https://example.com'  # Replace with your target URL
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')

links = []
for a_tag in soup.find_all('a', href=True):
    links.append(a_tag['href'])

print(links)

2. Using JavaScript in Browser Console

Open the browser console (F12), then run:

javascript
CopyEdit
const links = Array.from(document.querySelectorAll('a'))
                   .map(a => a.href);
console.log(links);

3. Using Command Line with wget and grep

bash
CopyEdit
wget -qO- https://example.com | grep -oP '(?<=href=")[^"]*'

If you want me to extract links from specific URLs, please share the URLs and I can do it for you.