Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.mapademics.com/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

Background jobs in Mapademics handle intensive processing tasks like analyzing course syllabi, extracting skills from job descriptions, and mapping occupational classifications. The platform provides real-time monitoring tools that let you track job progress, identify issues, and ensure your data processing completes successfully.

​
What You’ll Need

  • Administrative access to your organization
  • Basic understanding of your organization’s data processing workflows
  • Real-time connection indicator showing green status (🟢 Real-time)

​
Types of Background Jobs

Mapademics processes three main types of background jobs:

​
Syllabus Processing Jobs

Purpose: Extract skills and learning outcomes from course section syllabi
  • Process uploaded PDF, Word, or HTML syllabus files
  • Use AI to identify skills, competencies, and learning objectives
  • Update course sections with extracted skills data
  • Support batch processing of multiple syllabi simultaneously

​
Job Description Processing Jobs

Purpose: Analyze custom job descriptions to extract required skills
  • Process uploaded job description files
  • Identify technical skills, soft skills, and job requirements
  • Map skills to your custom job positions
  • Enable skills gap analysis between education and employment

​
SOC Classification Jobs

Purpose: Map Standard Occupational Classification codes to skills
  • Process government occupational data
  • Extract skills associated with specific job classifications
  • Build comprehensive skills databases for career mapping
  • Support national labor market intelligence

​
Monitoring Job Progress

​
Real-time Job Dashboard

  1. Access Background Jobs
    • Navigate to Background Jobs in your admin sidebar
    • Use the tabs at the top of the page to switch between job types:
      • Syllabi - Course syllabus analysis jobs
      • Jobs - Custom job description analysis jobs
      • Skills Mapping - SOC classification jobs
  2. Job Overview Table Your dashboard shows all jobs with key information:
    • Job Name: Unique identifier and Trigger.dev ID
    • Status: Current processing state with visual indicators
    • Progress: Real-time completion percentage with progress bar
    • Items: Total items and completion counts
    • Created: When the job was initiated
    • Actions: View detailed job information

​
Understanding Job Statuses

Processing States:
  • PENDING ⏳: Job is queued and waiting to start
  • PROCESSING 🔄: Job is actively running (animated spinner)
  • COMPLETED ✅: All items processed successfully
  • FAILED ❌: Job encountered unrecoverable errors
  • RETRYING 🔁: Job is attempting to recover from temporary failures
Progress Indicators:
  • Progress Bar: Visual representation of completion percentage
  • Item Counts: “Processed / Total Items” with failure counts
  • Real-time Updates: Automatic refresh as jobs progress

​
Detailed Job Monitoring

​
Viewing Individual Job Details

  1. Select a Job
    • Click View button next to any job in your dashboard
    • Or click directly on the job name
  2. Job Overview Section The detailed view shows:
    • Overall Status: Current job state with visual status indicator
    • Progress Metrics: Detailed completion statistics
    • Processing Statistics:
      • Total items to process
      • Currently processing count
      • Successfully completed count
      • Failed items count
  3. Individual Item Tracking For each processing item, you’ll see:
    • Course/Job Information: Which course section or job description is being processed
    • Item Status: Current processing state for that specific item
    • Error Details: Specific error messages if processing fails
    • Timestamps: When each item was processed or failed

​
Real-time Progress Updates

Automatic Refresh:
  • Progress bars update in real-time without page refresh
  • Status changes appear immediately when jobs complete or fail
  • Connection status indicator shows your real-time connection health
Progress Notifications:
  • Browser notifications for job completion (if enabled)
  • Visual status changes in the dashboard
  • Updated timestamps showing last activity

​
Job Performance Monitoring

​
Understanding Processing Speed

Normal Processing Times:
  • Single Syllabus: 30-60 seconds per document
  • Batch Jobs: Depends on total items and complexity
  • Job Descriptions: 20-40 seconds per description
  • SOC Classifications: Variable based on data volume
Performance Factors:
  • Document Complexity: Longer, more complex documents take more time
  • System Load: Processing may slow during peak usage
  • AI Processing: LLM analysis requires computational resources
  • Network Connectivity: Your connection affects real-time updates

​
Identifying Performance Issues

Warning Signs:
  • Jobs stuck in PROCESSING state for extended periods
  • High failure rates in batch processing
  • Repeated RETRYING status without progress
  • Missing real-time connection (🔴 Disconnected)
Performance Metrics to Watch:
  • Processing Rate: Items completed per minute
  • Failure Rate: Percentage of failed items
  • Queue Length: Number of pending jobs
  • Response Time: Time from job creation to completion

​
Troubleshooting Job Issues

​
Common Job Failures

File Access Problems:
  • Symptoms: Jobs fail immediately with file-related errors
  • Causes: Corrupted uploads, unsupported file formats, missing files
  • Solutions:
    • Re-upload problem files in supported formats (PDF, DOC, DOCX, HTML)
    • Verify files aren’t password-protected or corrupted
    • Check file size limits (typically 10MB maximum)
Processing Timeouts:
  • Symptoms: Jobs remain in PROCESSING state indefinitely
  • Causes: Complex documents, system overload, connectivity issues
  • Solutions:
    • Wait for automatic retry (system retries up to 3 times)
    • Check your internet connection stability
    • Try processing smaller batches
    • Contact support for persistent timeouts
AI Processing Errors:
  • Symptoms: Specific items fail with “processing error” messages
  • Causes: Unusual document formats, insufficient content, AI service issues
  • Solutions:
    • Review failed documents for unusual formatting
    • Ensure documents contain sufficient text content
    • Retry processing during off-peak hours
    • Process failed items individually to identify specific issues

​
Recovery and Retry Options

Automatic Recovery:
  • System automatically retries failed items up to 3 times
  • Exponential backoff prevents system overload
  • Partial failures don’t stop entire batch processing
Manual Recovery:
  • Failed items can be reprocessed individually
  • Re-upload corrected files and create new jobs
  • Use smaller batch sizes for problematic content
  • Contact support for persistent failures across multiple items

​
Connection and Real-time Issues

WebSocket Connection Problems:
  • Symptoms: 🔴 Disconnected or 🟡 Connecting status indicators
  • Solutions:
    • Refresh your browser page
    • Check your internet connection
    • Disable browser extensions that might block WebSockets
    • Try a different browser if issues persist
Missing Real-time Updates:
  • Symptoms: Job progress doesn’t update automatically
  • Solutions:
    • Look for connection status indicators
    • Manually refresh the page to see current status
    • Check browser console for WebSocket errors
    • Report persistent issues to technical support

​
Job History and Audit

​
Accessing Historical Data

Job Records:
  • All background jobs are permanently recorded
  • Access historical jobs from the main dashboard
  • Search by date, status, or job type
  • View detailed logs for completed and failed jobs
Audit Information:
  • Creation Timestamps: When jobs were initiated
  • Processing Times: Duration from start to completion
  • Error Logs: Detailed failure information
  • Configuration Used: Which AI processing settings were applied

​
Performance Analytics

Job Success Rates:
  • Track completion percentages over time
  • Identify patterns in processing failures
  • Monitor system performance trends
  • Compare processing speeds across different content types
Usage Patterns:
  • Peak processing times and system load
  • Most common job types in your organization
  • Average processing volumes per day/week/month
  • Seasonal variations in processing needs

​
Best Practices for Job Monitoring

​
Proactive Monitoring

Regular Check-ins:
  • Review job dashboard daily during active processing periods
  • Monitor real-time connection status
  • Watch for any jobs stuck in PROCESSING state
  • Address failed items promptly to prevent backlogs
Performance Optimization:
  • Optimal Batch Sizes: Process 10-20 items per batch for best performance
  • Off-Peak Processing: Schedule large jobs during low-usage periods
  • File Preparation: Ensure documents are clean, text-readable PDFs or Word files
  • Network Stability: Use reliable internet connections for large processing jobs

​
Error Prevention

File Quality Checks:
  • Verify all files open correctly before uploading
  • Remove password protection from documents
  • Ensure text is selectable (not just scanned images)
  • Use standard file formats (PDF, DOC, DOCX, HTML)
Batch Management:
  • Start with small test batches to verify processing
  • Group similar document types together
  • Avoid mixing very large and very small files in the same batch
  • Process urgent items separately from large bulk operations

​
Escalation Procedures

When to Contact Support:
  • Jobs consistently fail across multiple attempts
  • System shows persistent connection issues
  • Processing times exceed normal ranges significantly
  • Data integrity concerns after processing completion
Information to Provide:
  • Specific job IDs experiencing problems
  • Error messages from failed items
  • Screenshots of job status screens
  • Description of files being processed
  • Timeline of when issues began