Activate this automation on N8nPort with one click
We have completed the workflow installation, node configuration, and setup for you, so the automation is ready to activate without a manual import or node-by-node configuration.
Quick answers
What does this workflow do?
This workflow automatically scrapes content from specified web pages, extracts key information like titles and body text, and then uses an AI model to generate summaries for each page.
Can I use this workflow for any website?
This workflow is designed for publicly accessible web pages. You might need to adjust its CSS selectors if the target website's structure differs from the example, and it does not inherently handle websites requiring login.
What AI model does this workflow use?
The source workflow is configured to use an OpenAI chat model, specifically gpt-4o-mini, for its summarization tasks.
What credentials are needed to run this workflow?
You will need an OpenAI API Key to allow the workflow to interact with OpenAI's AI models for summarization. Additional service-specific API credentials might be needed if you customize the workflow for authenticated sites.
This n8n workflow offers a practical way to automate the process of extracting content from web pages and generating concise summaries using artificial intelligence. It's designed to help you quickly digest information from multiple online sources, such as articles, essays, or blog posts, without the need for manual reading.
Key Questions About This Workflow
What does this workflow do?
This workflow automatically scrapes content from specified web pages, extracts key information like titles and body text, and then uses an AI model to generate summaries for each page.
Can I use this workflow for any website?
This workflow is designed for publicly accessible web pages. You might need to adjust its CSS selectors if the target website's structure differs from the example, and it does not inherently handle websites requiring login.
What AI model does this workflow use?
The source workflow is configured to use an OpenAI chat model, specifically gpt-4o-mini, for its summarization tasks.
What credentials are needed to run this workflow?
You will need an OpenAI API Key to allow the workflow to interact with OpenAI's AI models for summarization. Additional service-specific API credentials might be needed if you customize the workflow for authenticated sites.
Quick Overview: Automate Webpage Scraping and AI Summarization
This n8n workflow automates the process of extracting and summarizing content from web pages using artificial intelligence. It fetches a list of articles, retrieves their full text, and then employs an OpenAI model to generate concise summaries. The final output provides the title, summary, and URL for each processed page, making it easier to review information efficiently.
What This Workflow Does
This n8n workflow automates the process of extracting and summarizing content from web pages using artificial intelligence. Specifically, it performs the following actions:
Fetches a list of articles or links from a specified starting URL.
Extracts specific links, such as individual essay URLs, from the fetched page.
Limits the number of items to process, for example, to the first three found.
Retrieves the full text content of each selected webpage.
Extracts the main title of each webpage.
Prepares the extracted text for AI processing by splitting it into manageable chunks.
Uses an AI model to generate a concise summary for each webpage.
Combines the original title, the AI-generated summary, and the source URL into a structured output.
How it works
This workflow automates the process of extracting and summarizing web content. It starts with a manual trigger and proceeds through several stages of data retrieval, processing, and AI interaction.
1. Webpage Content Retrieval
The workflow begins when you manually execute it. An HTTP Request node first fetches the HTML content from a specified URL, which is typically a page listing other articles (e.g., http://www.paulgraham.com/articles.html). Following this, an HTML node extracts specific links (e.g., essay URLs) from the fetched page using CSS selectors. A Split Out node then separates these links into individual items for processing.
To manage the volume, a Limit node is included, which, in the source workflow, restricts processing to the first three identified links. You can adjust or remove this limit as needed. For each selected link, another HTTP Request node retrieves the full HTML content of that specific webpage.
2. Text Extraction and Preparation
Once the individual webpage HTML is retrieved, two HTML nodes work to isolate the relevant text. One extracts the page's title using the title tag. The other, named "Extract Text Only," focuses on extracting the main body text, often excluding elements like images or navigation to ensure cleaner content for summarization.
The extracted text is then passed to a Document Default Data Loader node, which prepares the data for AI processing. For longer documents, a Recursive Character Text Splitter node breaks the text into smaller, manageable chunks (e.g., 6000 characters). This helps the AI model process extensive content efficiently and within its token limits.
3. AI Summarization
The prepared text chunks are sent to the OpenAI Chat Model node, which acts as the interface to OpenAI's AI services. This node requires an OpenAI API key for authentication and specifies the AI model to use (e.g., gpt-4o-mini). The Summarization Chain node then coordinates with the OpenAI Chat Model to generate a concise summary for each text chunk or the entire document, depending on its configuration.
4. Final Output Assembly
After summarization, a Merge node combines the extracted webpage title with its corresponding AI-generated summary. Finally, a Set node, labeled "Clean up," organizes this information into a structured output. This output typically includes the original webpage title, the new AI summary, and the source URL, making the results easy to review and use.
Ready to automate?
The complete installation and configuration are already prepared for your workspace. Activate the preconfigured workflow on N8nPort with one click.
Prerequisites
To run this workflow, you need access to specific n8n nodes and credentials:
Required Applications (n8n Nodes):
Chain Summarization: Manages the summarization process.
Document Default Data Loader: Prepares document data for AI.
HTML: Parses web pages and extracts content.
HTTP Request: Fetches web page content.
Lm Chat Open Ai: Interfaces with OpenAI's chat models.
Split Out: Processes items individually.
Text Splitter Recursive Character Text Splitter: Breaks large texts into smaller chunks.
Required Credentials:
OpenAI API Key: Essential for the OpenAI Chat Model node to connect with OpenAI services. Obtain this key from your OpenAI account and provide it when setting up the credential in n8n.
Service-specific API credential (if configured): If you modify the workflow to scrape authenticated websites or integrate with other services, you might need additional API credentials for those specific services. The core workflow primarily relies on the OpenAI API key.
Ensure these are configured in your n8n instance before activating the workflow.
How to Set It Up
Setting up this workflow involves importing it into your n8n instance and configuring the necessary credentials and node settings.
Import the Workflow: Download the workflow JSON from the source URL below. In n8n, go to "Workflows," click "New," then "Import from JSON," and paste the code.
Configure Credentials:
Locate the OpenAI Chat Model node.
Select or create a new OpenAI API Key credential and enter your API key. This key enables AI summarization.
Configure any other service-specific API credentials if you customize the workflow to scrape authenticated targets.
Review Node Settings:
Check HTTP Request nodes to ensure URLs match your scraping targets. The source targets
http://www.paulgraham.com/articles.htmland individual essays.Adjust HTML node CSS selectors if scraping different website structures.
Modify the "Max Items" in the Limit node to control the number of articles processed.
Review the
chunkSizein the Recursive Character Text Splitter node based on content length and AI model token limits.The OpenAI Chat Model node allows you to select a different OpenAI model if desired.
Save and Activate: After configuring settings and credentials, save and activate the workflow.
What You Can Customize
This workflow offers several customization options to fit your specific needs:
Target Websites: Change the URL in the first HTTP Request node to scrape different article lists or web pages. Adjust CSS selectors in HTML nodes to match the new site's structure.
Number of Items: Modify the "Max Items" value in the Limit node to control how many articles are processed.
Content Extraction: Adjust CSS selectors in HTML nodes to target different elements (e.g., authors, dates, specific text sections).
AI Model: In the OpenAI Chat Model node, select a different OpenAI model (e.g.,
gpt-3.5-turbo,gpt-4) based on quality and cost requirements.Text Splitting: Adjust the
chunkSizein the Recursive Character Text Splitter node for optimal processing of long documents.Output Structure: The final Set node ("Clean up") allows you to add or remove output fields (title, summary, url) to tailor the data structure.
Trigger Method: Replace the manual trigger with a schedule-based trigger for automated, regular scraping and summarization.
Limitations and Troubleshooting
Consider these points when using this workflow:
Website Structure Changes: Changes to a target website's HTML can break CSS selectors in HTML nodes, requiring updates.
Rate Limits: Be aware of rate limits from target websites and the OpenAI API to avoid temporary blocks or errors.
Content Length: Very long articles might still challenge summarization, even with text splitting, depending on the AI model and desired summary quality.
AI Cost: Using AI models incurs costs based on usage. Monitor your OpenAI API usage.
Credential Expiry: Ensure your OpenAI API key remains valid.
Error Handling: The workflow lacks explicit error handling. For useful use, consider adding nodes to manage errors.
If issues arise, check n8n execution logs for error messages. Verify credentials, review URLs and CSS selectors, and test individual nodes to diagnose problems.
Frequently Asked Questions
Is this workflow suitable for real-time summarization?
This workflow is designed for on-demand or scheduled processing. It is not typically configured for real-time, instantaneous summarization of live content without further modifications.
Can I use a different AI model than OpenAI?
The source workflow uses OpenAI. To use another AI model, you would need to replace the OpenAI Chat Model node with a compatible node for your chosen service and configure its specific credentials.
How can I save the summaries?
The workflow outputs summaries within n8n. To save them, add nodes after "Clean up," such as "Write to File," "Google Sheets," "Database," or "Email" nodes.
What if the website I want to scrape requires login?
This workflow is for publicly accessible pages. For websites requiring login, you would need to add nodes to handle authentication (e.g., sending credentials via HTTP Request), which can be complex depending on the site's method.
Source and Next Steps
This workflow was originally shared on the n8n.io website. You can find the original workflow and more details here: Scrape and summarize webpages with AI on n8n.io.
To explore further, consider adapting this workflow to summarize content from your favorite blogs, news sites, or research papers. Experiment with different AI models and text splitting strategies to optimize the summarization quality for your specific use cases.
Activate this workflow on N8nPort
Choose your workspace and activate the fully prepared automation with one click.