Scraping real estate data with Python to find opportunities

This tutorial will explore how to use Python’s requests library to scrape real estate data from an API. We'll also learn how to apply filters to retrieve potentially bargain properties that have recently had their prices reduced.

---

Introduction

When looking for great real estate investment opportunities, recent price reductions are often one of the most important indicators. Having a tool that displays these properties quickly can save a lot of time and may help you get a head start before anyone else notices!

In this article we will:

1. Discuss the basics of interacting with the real estate API using requests.
2. Learn how to use query parameters to filter results—especially focusing on price change queries.
3. Parse and display returned data in a concise format.

---

Requirements

• InstalledPython 3
• Terminal or command line prompt
• Familiar with the basics of the Python requests library
• API key (if required by API)

---

Step 1: Understand the API

The API we use may return the following data:

• Property ID
• Title or address
• Price
• Location
• Historical Price Changes
• Other related information

Key query parameters

This API supports multiple query parameters that help us filter results:

参数	类型	描述
**includedDepartments[]**	数组	按部门过滤。示例：departments/77
**fromDate**	日期	仅检索在此日期之后列出（或更新）的房产。
**propertyTypes[]**	数组	按房产类型过滤。示例：0代表公寓，1代表房屋，等等。
**transactionType**	字符串	0代表出售，1代表出租，等等。
**withCoherentPrice**	布尔值	仅检索价格与市场价格一致的房产。
**budgetMin**	数字	最低预算阈值。
**budgetMax**	数字	最高预算阈值。
**eventPriceVariationFromCreatedAt**	日期	创建价格类型事件的日期——包含在内。
**eventPriceVariationMin**	数字	价格变化的最小百分比（负数或正数）。

We will pay special attention to the **eventPriceVariation** parameter to **find properties** that have dropped in price.

---

Step 2: Create Request

The following is an example script for querying an endpoint using Python's requests library. Adjust parameters and headers as needed, especially if X-API-KEY is required.

import requests
import json

# 1. 定义端点URL
url = "https://api.stream.estate/documents/properties"

# 2. 创建参数
params = {
    'includedDepartments[]': 'departments/77',
    'fromDate': '2025-01-10',
    'propertyTypes[]': '1',    # 1可能代表“公寓”
    'transactionType': '0',    # 0可能代表“出售”
    'withCoherentPrice': 'true',
    'budgetMin': '100000',
    'budgetMax': '500000',
    # 关注价格变化
    'eventPriceVariationFromCreatedAt': '2025-01-01',  # 从年初开始
    'eventPriceVariationMin': '-10',  # 至少下降10%
}

# 3. 使用API密钥定义标头
headers = {
  'Content-Type': 'application/json',
  'X-API-KEY': '<your_api_key_here>'
}

# 4. 发出GET请求
response = requests.get(url, headers=headers, params=params)

# 5. 处理响应
if response.status_code == 200:
    data = response.json()
    print(json.dumps(data, indent=2))
else:
    print(f"请求失败，状态码为{response.status_code}")

Important parameter description

eventPriceVariationMin = '-10'

This means you are looking for a price drop of at least 10%.

eventPriceVariationMax = '0'

Setting this to 0 ensures that you do not include properties that have experienced price increases or any changes above 0%. Essentially, you are capturing negative or zero change.

? Tip: Adjust the min/max values ​​to suit your strategy. For example, -5 and 5 would include price changes within ±5%.

Potential pitfalls and precautions

1. Authentication: Always make sure you use a valid API key. Some APIs also have rate limits or usage quotas.
2. Error handling: Handle situations where API is down or parameters are invalid.
3. Data Validation: The API may return incomplete data for some lists. Always check for missing fields.
4. Date Format: Make sure your fromDate and toDate are in a format recognized by the API (e.g., YYYY-MM-DD).
5. Large Datasets: If the API returns hundreds or thousands of lists, pagination may be required. Check whether paging parameters such as page or limit exist in the API document.

---

Summary

Now you have a basic Python script to crawl real estate data, focusing on properties that have dropped in price. This method can be very powerful if you want to invest in real estate, or just want to track market trends.

As always, please adjust the parameters to your specific needs. You can extend this script to sort results by price, integrate advanced analytics, and even plug data into a machine learning model for deeper insights.

Happy hunting and may you find hidden gems!

---

Further reading

• Python Requests Documentation
• Real Estate Data API Comparison
• Stream Estate API
• Key Points of Real Estate Data API

The above is the detailed content of Scraping real estate data with Python to find opportunities. For more information, please follow other related articles on the PHP Chinese website!