Alternative ways to transfer records from one environment to another

Question

I'm working on an application which has a feature of syncing records between two environments. For example, a record A is created in environment A. After a user verifies it, the user can use the sync feature to transfer the record from environment A to B. The solution has been implemented using Django API and requests module. The environment file consists of the two variables, currEnv and deployEnv. Based on their values in each environment, the records would either be sent to another environment or stored in the database.

Sample code:

import requests;
class SyncRecord():
    def post(self,request):
        if env.currEnv!=env.deployEnv:
            requests.post('http://www.example.com',json.loads(request.POST))
        else:
            Model.save(request.POST)

This is currently working, but I was looking for other ways to do it. I have considered using queuing systems like Kafka and SNS, but couldn't apply to my service as there is neither a producer nor a consumer. The current solution is taking a lot of time since it needs to transfer from one environment to another and pass through different conditions.

What changes could be done to improve my API?

Answer 1

It's unclear why this takes a "long" time:

    requests.post('http://www.example.com', json.loads(request.POST))

If you're looking for async bulk solutions, they are easy to implement.

(1.) Write the request to data/YYYY-MM-DD/HH-MM-SS-GUID.json, and use rsync -av data/ someserver:/data/ to send recent updates to another server. Or focus on a daily directory, to speed things up.

(2.) INSERT each request to your record table, which has an indexed updated timestamp column. Then sending recent updates is a matter of SELECT * FROM record WHERE updated > recent, and doing an INSERT of such records into another server's table.