REST (Representational State Transfer) APIs have become the backbone of modern web applications and data science workflows. As a data scientist, understanding how to consume and create REST APIs is essential for accessing external data sources, deploying machine learning models, and building scalable data applications.
In this chapter we explore why REST exists and its fundamental principles, how REST builds on HTTP and JSON, practical examples using real APIs, and the building and deploying of a REST API for model scoring.
The Problem REST Solves
Before REST, different systems had various ways of communicating over the internet, often complex and proprietary. REST emerged to solve several key problems (R. T. Fielding 2000):
Standardization: Need for a common, predictable way for systems to communicate
Scalability: Ability to handle millions of requests efficiently
Simplicity: Easy to understand and implement
Platform Independence: Works across different programming languages and systems
Stateless: Each request contains all information needed to process it
Client-Server Architecture: Clear separation between data consumer and provider
Cacheable: Responses can be cached to improve performance
Uniform Interface: Consistent way to interact with resources
Layered System: Architecture can have multiple layers (proxies, gateways, etc.)
Code on Demand (optional): Server can send executable code to client
35.2 REST is Built on HTTP and JSON
REST leverages two foundational web technologies: HTTP (HyperText Transfer Protocol) for communication and JSON (JavaScript Object Notation) for data exchange (Richardson and Ruby 2007).
HTTP: The Communication Protocol
HTTP is the protocol that powers the web. It defines how messages are formatted and transmitted between clients and servers.
Key HTTP Components:
URL (Uniform Resource Locator): Identifies the resource
Among the JSON advantages are its support by all programming languages, it is easy to read and write, compact and efficient.
35.3 Understanding HTTP Methods: GET and POST
HTTP GET Method
GET is used to retrieve data from a server. It is safe and idempotent (multiple identical requests have the same effect) (R. Fielding and Reschke 2014b).
Characteristics
Data are sent in URL parameters
Limited data size (URL length limits)
Cacheable
Should not modify server state
Example GET Request
In this example we use the popular OpenWeatherMap API to retrieve real-world, real-time weather data via a REST API (OpenWeather Ltd 2025). Before you can retrieve weather data through this service, sign up at OpenWeatherMap and retrieve a free API key. You will need to submit the key as part of the API request—that is how OpenWeatherMap keeps track of your usage. The first 1,000 calls per day are free.
The API key is a string, and it is not recommended to include the string directly into your program. When code is shared you do not want any API keys to get away from you. The recommended way is to store the API key in an environment variable and to retrieve it in real time in the program. This can be problematic sometimes, depending on your environment. An environment variable set in one shell might not be visible from the Python process, depending on how it was started.
To work around it, you can store the environment variables in a file named .env in your local directory, and use the load_dotenv method in the dotenv library to retrieve the variable. The following code does exactly that.
If you keep the .env file in the same directory as a repository that is managed with git, add the .env file to your .gitignore file to prevent accidental inclusion in a remote repository.
POST is used to send data to a server, typically to create new resources or submit data for processing.
Characteristics
Data are sent in request body
No size limitations
Not cacheable
Can modify server state
Example POST Request
We are sending in this example a simple request to httpbin.org, a simple HTTP request & response service. httpbin.org is a free, online service that provides a variety of HTTP endpoints for testing and debugging HTTP clients and libraries. It is essentially a “meta API” that allows users to send requests and inspect the responses.
import requestsimport json# POST request to submit dataurl ="https://httpbin.org/post"data = {"name": "John Doe","email": "john@example.com","message": "Hello from Python!"}response = requests.post(url, json=data)print(response.json())
# test_api.pyimport requestsimport jsonimport numpy as np# API base URLBASE_URL ="http://localhost:5000"def test_health_check():"""Test the health check endpoint""" response = requests.get(f"{BASE_URL}/")print("Health Check:")print(json.dumps(response.json(), indent=2))print()def test_model_info():"""Test the model info endpoint""" response = requests.get(f"{BASE_URL}/model-info")print("Model Info:")print(json.dumps(response.json(), indent=2))print()def test_single_prediction():"""Test single prediction"""# Generate random features (in practice, use real data) features = np.random.randn(10).tolist() data = {"features": features } response = requests.post(f"{BASE_URL}/predict", json=data, headers={'Content-Type': 'application/json'} )print("Single Prediction:")print(json.dumps(response.json(), indent=2))print()def test_batch_prediction():"""Test batch prediction"""# Generate multiple random feature sets batch_data = []for i inrange(3): features = np.random.randn(10).tolist() batch_data.append({"features": features}) response = requests.post(f"{BASE_URL}/predict", json=batch_data, headers={'Content-Type': 'application/json'} )print("Batch Prediction:")print(json.dumps(response.json(), indent=2))print()if__name__=="__main__":print("Testing Random Forest API...\n") test_health_check() test_model_info() test_single_prediction() test_batch_prediction()
Step 4: Enhanced API with Input Validation
# enhanced_app.pyfrom flask import Flask, request, jsonifyfrom marshmallow import Schema, fields, ValidationErrorimport joblibimport numpy as npimport pandas as pdfrom datetime import datetimeimport logging# Set up logginglogging.basicConfig(level=logging.INFO)logger = logging.getLogger(__name__)# Input validation schemasclass PredictionSchema(Schema): features = fields.List(fields.Float(), required=True)class BatchPredictionSchema(Schema): predictions = fields.List(fields.Nested(PredictionSchema), required=True)app = Flask(__name__)app.config['JSON_SORT_KEYS'] =False# Load modeltry: model = joblib.load('random_forest_model.pkl') feature_names = joblib.load('feature_names.pkl') logger.info("Model loaded successfully")exceptExceptionas e: logger.error(f"Error loading model: {str(e)}") model =None feature_names =None@app.route('/predict', methods=['POST'])def predict_enhanced():"""Enhanced prediction endpoint with validation"""if model isNone:return jsonify({'error': 'Model not available','status': 'error' }), 503try: data = request.get_json(force=True)# Validate input dataif'features'in data:# Single prediction schema = PredictionSchema() validated_data = schema.load(data) predictions = [make_prediction_with_validation(validated_data['features'])]else:# Batch prediction batch_schema = BatchPredictionSchema() validated_data = batch_schema.load({'predictions': data}) predictions = [ make_prediction_with_validation(item['features']) for item in validated_data['predictions'] ]return jsonify({'predictions': predictions,'status': 'success','timestamp': datetime.now().isoformat(),'count': len(predictions) })except ValidationError as e:return jsonify({'error': 'Validation failed','details': e.messages,'status': 'error' }), 400exceptExceptionas e:return jsonify({'error': str(e),'status': 'error' }), 500def make_prediction_with_validation(features):"""Make prediction with input validation"""iflen(features) !=len(feature_names):raiseValueError(f"Expected {len(feature_names)} features, received {len(features)}")# Check for invalid valuesifany(notisinstance(f, (int, float)) or np.isnan(f) or np.isinf(f) for f in features):raiseValueError("Features must be finite numeric values") X = np.array(features).reshape(1, -1) prediction = model.predict(X)[0] probabilities = model.predict_proba(X)[0]return {'prediction': int(prediction),'probabilities': {f'class_{i}': float(prob) for i, prob inenumerate(probabilities) },'confidence': float(max(probabilities)) }if__name__=='__main__': app.run(debug=True, host='0.0.0.0', port=5000)
# Install dependenciespip install flask scikit-learn joblib marshmallow numpy pandas# Run the applicationpython app.py# Test in another terminalpython test_api.py
Production Deployment Options
1. Using Gunicorn (Recommended for Production)
# Install Gunicornpip install gunicorn# Create requirements.txtpip freeze > requirements.txt# Run with Gunicorngunicorn --bind 0.0.0.0:5000 --workers 4 app:app
Fielding, Roy, and Julian Reschke. 2014a. “Hypertext Transfer Protocol (HTTP/1.1): Message Syntax and Routing.” RFC 7230. Internet Engineering Task Force (IETF). https://doi.org/10.17487/RFC7230.
———. 2014b. “Hypertext Transfer Protocol (HTTP/1.1): Semantics and Content.” RFC 7231. Internet Engineering Task Force (IETF). https://doi.org/10.17487/RFC7231.