Overview
Background jobs in Mapademics handle intensive processing tasks like analyzing course syllabi, extracting skills from job descriptions, and mapping occupational classifications. The platform provides real-time monitoring tools that let you track job progress, identify issues, and ensure your data processing completes successfully.What You’ll Need
- Administrative access to your organization
- Basic understanding of your organization’s data processing workflows
- Real-time connection indicator showing green status (🟢 Real-time)
Types of Background Jobs
Mapademics processes three main types of background jobs:Syllabus Processing Jobs
Purpose: Extract skills and learning outcomes from course section syllabi- Process uploaded PDF, Word, or HTML syllabus files
- Use AI to identify skills, competencies, and learning objectives
- Update course sections with extracted skills data
- Support batch processing of multiple syllabi simultaneously
Job Description Processing Jobs
Purpose: Analyze custom job descriptions to extract required skills- Process uploaded job description files
- Identify technical skills, soft skills, and job requirements
- Map skills to your custom job positions
- Enable skills gap analysis between education and employment
SOC Classification Jobs
Purpose: Map Standard Occupational Classification codes to skills- Process government occupational data
- Extract skills associated with specific job classifications
- Build comprehensive skills databases for career mapping
- Support national labor market intelligence
Monitoring Job Progress
Real-time Job Dashboard
-
Access Background Jobs
- Navigate to Background Jobs in your admin sidebar
- Use the tabs at the top of the page to switch between job types:
- Syllabi - Course syllabus analysis jobs
- Jobs - Custom job description analysis jobs
- Skills Mapping - SOC classification jobs
-
Job Overview Table
Your dashboard shows all jobs with key information:
- Job Name: Unique identifier and Trigger.dev ID
- Status: Current processing state with visual indicators
- Progress: Real-time completion percentage with progress bar
- Items: Total items and completion counts
- Created: When the job was initiated
- Actions: View detailed job information
Understanding Job Statuses
Processing States:- PENDING ⏳: Job is queued and waiting to start
- PROCESSING 🔄: Job is actively running (animated spinner)
- COMPLETED ✅: All items processed successfully
- FAILED ❌: Job encountered unrecoverable errors
- RETRYING 🔁: Job is attempting to recover from temporary failures
- Progress Bar: Visual representation of completion percentage
- Item Counts: “Processed / Total Items” with failure counts
- Real-time Updates: Automatic refresh as jobs progress
Detailed Job Monitoring
Viewing Individual Job Details
-
Select a Job
- Click View button next to any job in your dashboard
- Or click directly on the job name
-
Job Overview Section
The detailed view shows:
- Overall Status: Current job state with visual status indicator
- Progress Metrics: Detailed completion statistics
- Processing Statistics:
- Total items to process
- Currently processing count
- Successfully completed count
- Failed items count
-
Individual Item Tracking
For each processing item, you’ll see:
- Course/Job Information: Which course section or job description is being processed
- Item Status: Current processing state for that specific item
- Error Details: Specific error messages if processing fails
- Timestamps: When each item was processed or failed
Real-time Progress Updates
Automatic Refresh:- Progress bars update in real-time without page refresh
- Status changes appear immediately when jobs complete or fail
- Connection status indicator shows your real-time connection health
- Browser notifications for job completion (if enabled)
- Visual status changes in the dashboard
- Updated timestamps showing last activity
Job Performance Monitoring
Understanding Processing Speed
Normal Processing Times:- Single Syllabus: 30-60 seconds per document
- Batch Jobs: Depends on total items and complexity
- Job Descriptions: 20-40 seconds per description
- SOC Classifications: Variable based on data volume
- Document Complexity: Longer, more complex documents take more time
- System Load: Processing may slow during peak usage
- AI Processing: LLM analysis requires computational resources
- Network Connectivity: Your connection affects real-time updates
Identifying Performance Issues
Warning Signs:- Jobs stuck in PROCESSING state for extended periods
- High failure rates in batch processing
- Repeated RETRYING status without progress
- Missing real-time connection (🔴 Disconnected)
- Processing Rate: Items completed per minute
- Failure Rate: Percentage of failed items
- Queue Length: Number of pending jobs
- Response Time: Time from job creation to completion
Troubleshooting Job Issues
Common Job Failures
File Access Problems:- Symptoms: Jobs fail immediately with file-related errors
- Causes: Corrupted uploads, unsupported file formats, missing files
- Solutions:
- Re-upload problem files in supported formats (PDF, DOC, DOCX, HTML)
- Verify files aren’t password-protected or corrupted
- Check file size limits (typically 10MB maximum)
- Symptoms: Jobs remain in PROCESSING state indefinitely
- Causes: Complex documents, system overload, connectivity issues
- Solutions:
- Wait for automatic retry (system retries up to 3 times)
- Check your internet connection stability
- Try processing smaller batches
- Contact support for persistent timeouts
- Symptoms: Specific items fail with “processing error” messages
- Causes: Unusual document formats, insufficient content, AI service issues
- Solutions:
- Review failed documents for unusual formatting
- Ensure documents contain sufficient text content
- Retry processing during off-peak hours
- Process failed items individually to identify specific issues
Recovery and Retry Options
Automatic Recovery:- System automatically retries failed items up to 3 times
- Exponential backoff prevents system overload
- Partial failures don’t stop entire batch processing
- Failed items can be reprocessed individually
- Re-upload corrected files and create new jobs
- Use smaller batch sizes for problematic content
- Contact support for persistent failures across multiple items
Connection and Real-time Issues
WebSocket Connection Problems:- Symptoms: 🔴 Disconnected or 🟡 Connecting status indicators
- Solutions:
- Refresh your browser page
- Check your internet connection
- Disable browser extensions that might block WebSockets
- Try a different browser if issues persist
- Symptoms: Job progress doesn’t update automatically
- Solutions:
- Look for connection status indicators
- Manually refresh the page to see current status
- Check browser console for WebSocket errors
- Report persistent issues to technical support
Job History and Audit
Accessing Historical Data
Job Records:- All background jobs are permanently recorded
- Access historical jobs from the main dashboard
- Search by date, status, or job type
- View detailed logs for completed and failed jobs
- Creation Timestamps: When jobs were initiated
- Processing Times: Duration from start to completion
- Error Logs: Detailed failure information
- Configuration Used: Which AI processing settings were applied
Performance Analytics
Job Success Rates:- Track completion percentages over time
- Identify patterns in processing failures
- Monitor system performance trends
- Compare processing speeds across different content types
- Peak processing times and system load
- Most common job types in your organization
- Average processing volumes per day/week/month
- Seasonal variations in processing needs
Best Practices for Job Monitoring
Proactive Monitoring
Regular Check-ins:- Review job dashboard daily during active processing periods
- Monitor real-time connection status
- Watch for any jobs stuck in PROCESSING state
- Address failed items promptly to prevent backlogs
- Optimal Batch Sizes: Process 10-20 items per batch for best performance
- Off-Peak Processing: Schedule large jobs during low-usage periods
- File Preparation: Ensure documents are clean, text-readable PDFs or Word files
- Network Stability: Use reliable internet connections for large processing jobs
Error Prevention
File Quality Checks:- Verify all files open correctly before uploading
- Remove password protection from documents
- Ensure text is selectable (not just scanned images)
- Use standard file formats (PDF, DOC, DOCX, HTML)
- Start with small test batches to verify processing
- Group similar document types together
- Avoid mixing very large and very small files in the same batch
- Process urgent items separately from large bulk operations
Escalation Procedures
When to Contact Support:- Jobs consistently fail across multiple attempts
- System shows persistent connection issues
- Processing times exceed normal ranges significantly
- Data integrity concerns after processing completion
- Specific job IDs experiencing problems
- Error messages from failed items
- Screenshots of job status screens
- Description of files being processed
- Timeline of when issues began
Related Tasks
- Course Section Management - Manage syllabi and course content
- Custom Job Management - Upload and manage job descriptions
- Skills Mapping - SOC classification processing
- Organization Configuration - System settings and preferences