Skip to main content

shedboxai run Command

Execute your configured data processing pipelines with options for output and logging control.

Basic Usage​

shedboxai run config.yaml

Command Syntax​

shedboxai run [OPTIONS] CONFIG_FILE

Arguments​

  • CONFIG_FILE - Path to your YAML configuration file (required)

Options​

OptionShortDescriptionExample
--output-oSpecify output file path-o results.json
--verbose-vEnable detailed logging-v
--quiet-qSuppress log messages-q

Detailed Options​

Output Control​

--output / -o​

Specify where to save pipeline results:

# Save to JSON file
shedboxai run config.yaml -o results.json

# Save to custom path
shedboxai run config.yaml -o /path/to/results.json

Output Format:

  • All output is saved in JSON format regardless of file extension

Logging Control​

--verbose / -v​

Enable detailed logging for debugging:

shedboxai run config.yaml -v

Verbose Output Includes:

  • Data source connection details
  • Processing step execution
  • Record counts and transformations
  • Error details and stack traces

--quiet / -q​

Suppress all log messages except errors:

shedboxai run config.yaml -q

Quiet Mode:

  • Only critical errors are displayed
  • No informational messages
  • Faster execution (no logging overhead)

Usage Examples​

Basic Execution​

# Run pipeline with default settings
shedboxai run pipeline.yaml

# Run with verbose logging
shedboxai run pipeline.yaml --verbose

# Run quietly (errors only)
shedboxai run pipeline.yaml --quiet

Output Management​

# Save results to specific file
shedboxai run config.yaml -o analysis_results.json

# Save with timestamp
shedboxai run config.yaml -o "results_$(date +%Y%m%d_%H%M%S).json"

# Save to different directory
shedboxai run config.yaml -o output/daily_analysis.yaml

Combined Options​

# Verbose execution with custom output
shedboxai run config.yaml -v -o debug_results.json

# Quiet execution for production
shedboxai run config.yaml -q -o production_output.json

# Development debugging
shedboxai run config.yaml --verbose --output detailed_debug.yaml

Configuration File Requirements​

Valid Configuration Files​

The configuration file must be valid YAML with required sections:

# Minimum required configuration
data_sources:
  example:
    type: csv
    path: data.csv

output:
  type: print

Configuration Validation​

The command validates your configuration before execution:

  • YAML Syntax: Valid YAML format
  • Required Sections: Must have data_sources
  • Data Source Paths: Files must exist and be readable
  • API Credentials: Environment variables must be set
  • Output Paths: Directories must be writable

Exit Codes​

CodeMeaningDescription
0SuccessPipeline completed successfully
1ErrorAny error occurred during execution
130CancelledOperation cancelled by user (Ctrl+C)

Error Handling​

Common Errors and Solutions​

Configuration Errors​

Error: Invalid YAML syntax in config.yaml
Solution: Validate YAML syntax using online validator

Data Source Errors​

Error: File not found: data/users.csv
Solution: Check file path and ensure file exists

Permission Errors​

Error: Permission denied writing to output/results.json
Solution: Check directory permissions or run with appropriate privileges

API Errors​

Error: API authentication failed for OpenAI
Solution: Verify OPENAI_API_KEY environment variable is set

Debug Mode​

Use verbose mode to debug issues:

shedboxai run config.yaml -v -o debug.json 2>&1 | tee debug.log

This will:

  • Enable verbose logging (-v)
  • Save output to debug.json
  • Capture both stdout and stderr
  • Save log to debug.log file

Performance Considerations​

Large Datasets​

For large datasets, consider:

# Time execution
time shedboxai run config.yaml -o results.json

# Use verbose mode for detailed logging
shedboxai run config.yaml -v

Environment Variables​

Optional Variables​

Environment variables for API access may be required depending on your configuration:

# For AI-powered features (if configured)
export OPENAI_API_KEY="your_openai_key"
export ANTHROPIC_API_KEY="your_claude_key"

Automation​

Basic Script Usage​

Use in shell scripts:

#!/bin/bash
# Run pipeline with error handling
if shedboxai run config.yaml -q -o results.json; then
    echo "Pipeline completed successfully"
else
    echo "Pipeline failed with exit code $?"
fi

With Timeout​

# Limit execution time
timeout 300 shedboxai run config.yaml -o results.json  # 5 minutes max

Troubleshooting​

Common Issues​

  1. Configuration not found

    Error: Configuration file 'config.yaml' not found
    Solution: Verify file path and current directory

  2. Permission denied

    Error: Permission denied: Cannot write to output directory
    Solution: Check directory permissions or use different output path

  3. API rate limits

    Error: Rate limit exceeded for API calls
    Solution: Add delays in configuration or reduce batch size

  4. Memory issues

    Error: Memory allocation failed
    Solution: Process data in smaller chunks or increase available memory

Getting Help​

# Show command help
shedboxai run --help

# Enable verbose logging for debugging
shedboxai run config.yaml -v

Next Steps​