Scaling SEO Operations: How Automation Helped an Agency 10x Their Output
The Challenge
A specialized SEO consultancy faced growing demands from e-commerce clients needing large-scale optimization of product and collection pages. Their established process involved multiple manual steps:
- Extracting page data from SEMrush
- Analyzing page content and existing rankings
- Creating optimized titles and meta descriptions
- Managing data across multiple spreadsheets
- Coordinating between team members handling different process stages
"They do this off spreadsheets... nothing else," explains the project lead. "Somebody on their team does the SEMrush part, exports it, cleans it up... and then passes along that report to the person that does the actual analysis."
This manual workflow limited their capacity and created bottlenecks between team members.
The Solution
Working with Connex, the agency implemented an automated workflow using:
- Zapier for process automation
- OpenAI's API for content analysis and generation
- SEMrush API for ranking data
- Google Sheets for input/output management
The new system follows a sophisticated multi-stage process:
- Content Analysis: Automatically scrapes and analyzes page content, extracting key elements and brand voice
- Keyword Analysis: Processes SEMrush ranking data to identify optimization opportunities
- Title & Description Generation: Creates multiple AI-powered optimization options using both existing rankings and content analysis
- Output Review: Presents results in an organized format with color-coding for different optimization approaches
Implementation
The solution was developed iteratively over 3-4 weeks, with regular testing and refinement. Key focus areas included:
- Ensuring consistent structured data handling
- Refining AI prompts for accuracy
- Building reliable parsing systems
- Creating an intuitive interface for team members
How it works in detail
flowchart TB subgraph Input A[User pastes URLs in Input Tab] --> B[Zapier triggers workflow] end subgraph ContentAnalysis B --> C[Web scraping] C --> D[Clean HTML content] D --> E[Extract key elements] E --> F[AI Content Analysis] end subgraph KeywordAnalysis B --> G[SEMrush API call] G --> H[Sort keywords by ranking] G --> I[Sort keywords by traffic] G --> J[Sort keywords by search volume] H & I & J --> K[AI Keyword Selection] end subgraph Generation F & K --> L[Generate AI-based titles/desc] F & K --> M[Generate SEMrush-based titles/desc] L & M --> N[Generate combined titles/desc] end subgraph Output N --> O[Results in Output Tab] O --> P[Blue: AI-generated] O --> Q[Orange: SEMrush-based] O --> R[Gray: Combined refinement] end
Here's how the system works now:
- Input Process
- Users paste URLs into the input tab of a Google Sheet
- Processing takes about 5-6 minutes per link
- Users can paste multiple links (e.g., 20) and work on other tasks while processing
- Content Analysis Stage
- System scrapes each webpage
- Cleans up HTML by removing:
- Scripts
- Styling
- Footer
- Navigation
- Extracts key elements:
- Original title
- Meta description (if exists)
- H1 headers
- Main content
- AI analyzes content for:
- Page type (product/collection/etc.)
- Brand voice
- Unique selling propositions
- Writing style
- Keyword Analysis Stage
- Calls SEMrush API for each URL
- Gets organic keyword data:
- Current rankings
- Search volume
- Traffic data
- Sorts keywords in multiple ways:
- By ranking position
- By traffic volume
- By search volume
- Generation Process The system creates three sets of optimizations:
- Blue Output: Pure AI-generated
- Based on content analysis
- Content-aware but without ranking data
- Orange Output: SEMrush-based
- Uses existing ranking data
- Optimizes for current performing keywords
- Gray Output: Combined refinement
- Merges insights from both approaches
- Provides most relevant recommendations
- Output Format
- Each URL gets 15 total options:
- 5 title variations per approach (15 total)
- 5 description variations per approach (15 total)
- Color-coded for easy differentiation
- All results appear in the output tab
- Processing Time
- Average 5-6 minutes per URL
- Can handle batch processing of ~20 URLs at once
- Uses queues to maintain orderly processing
This system has dramatically improved the agency's efficiency, allowing them to process about 10 times more URLs than their previous manual process. The multiple optimization approaches (AI-only, SEMrush-based, and combined) give users flexibility in choosing the most appropriate optimization for each page while maintaining their unique SEO methodology that considers both existing rankings and new opportunities.
The Results
The automation delivered significant improvements in both efficiency and output:
- 10x increase in page optimization capacity
- Reduction in manual data handling
- Streamlined workflow between team members
- Consistent quality through standardized processes
"They can do like 10 times more links," notes the project lead. The system now enables team members to simply "paste like 20 links, go to something else, come back" and find complete optimization suggestions ready for review.
Unique Approach
The solution preserves the agency's distinctive SEO methodology, which considers both:
- Existing keyword rankings to maintain current performance
- New optimization opportunities based on content analysis
This balanced approach helps clients maintain existing search visibility while expanding their reach.
Looking Forward
While operating as an MVP, the system has demonstrated significant potential for scaling SEO operations. Future optimizations may include:
- Migration to more structured AI platforms
- Enhanced automation capabilities
- Additional optimization parameters
For SEO agencies facing similar scaling challenges, this case study demonstrates how strategic automation can dramatically increase capacity while maintaining quality and methodology consistency.
The bold text helps guide readers through the key points while maintaining the professional tone expected in the SEO industry. The structure remains clear and the emphasis helps highlight the most important aspects of each section.
