Ethereum Smart Contracts and the Oracle Problem

By their nature, smart contracts are SELF-CONTAINED scripts of code, meaning they don’t intrinsically have access to external information such as web APIs or filesystems. There’s a good reason for this: In the case of Ethereum, it’s all about determinism and verifiable data.

When someone successfully publishes a smart contract, It gets executed on every machine that maintains a full copy of the blockchain. It’s important that this execution remains consistent across all machines so that the same result is generated every time.

Otherwise, each Ethereum node would fail to reach a consensus over the current state. The internet is non-deterministic because it changes over time. Retrieving data from an external resource (such as a bitcoin price ticker) can and often will return a different result. This leaves us with the question, how do we aggregate non-deterministic data into a deterministic environment such as a smart contract?

Oracles bring real-world data into Blockchain

“Provable”, formerly known as “Oraclize”, is a service known as an ‘oracle’, built specifically to address this problem. An oracle acts as a relay by aggregating data from external sources such as random number generators, price tickers & computational engines. Once the data is collected, the oracle feeds it into a smart contract.

You may be wondering, doesn’t a third-party relay defeat the purpose of data decentralization? Well, you’d be absolutely correct. That’s why Provable implements the concept of authenticity proofs. Using secure cryptography, we’re able to verify that the information passing though has not been tampered with from the original source. Thus, the oracle can be trusted to deliver deterministic results to the deterministic environment.

Let’s examine a possible use case with an example. Perhaps you’re a landlord and you wish to collect deposits and rent in a safe & secure fashion. In this tutorial, we’ll create a means for landlords to draw up leases for tenants. Here’s how it works:

1. The landlord and tenant agree on rental terms in person (i.e. $1000/month for 6 months + deposit of $500. Each payment is due on the first of every month.
2. The landlord creates a lease in a smart contract, providing it with the terms and the tenant’s Ethereum address. All monetary amounts are conveyed as US dollars.
3. The tenant sends the initial deposit to the contract, which initiates the time limit for the first monthly lease payment.
4. The smart contract ensures the deposit and monthly payment amount is correct by converting the amount of Ether sent to US dollars using conversion data provided by Provable Oracle.
5. After the lease is finished, the tenant may reclaim the deposit.
6. If the tenant fails to make a monthly payment before the time limit, the landlord may revoke the deposit.
7. The landlord may deposit Ether and withdraw rental income at any time.

The only risk for the landlord is that the value in Ether may decrease over the rental period. In this case, more Ether must be deposited into the contract to satisfy tenants’ deposit refunds. It may increase though, providing a positive return.

Notice: This is an intermediate-level tutorial. A basic level of knowledge of Ethereum, Javascript, Solidity, and the Truffle framework is advised, but not totally necessary. If you’re just starting out, read my introductory article to provide some context. Furthermore, this guide is quite lengthy. As usual, code will be explained line by line to ensure proper understanding.

Setting Up The Environment

1. Ensure you have NodeJS installed, head over to their official website or install using HomeBrew.

brew install node

2. Ensure Truffle is installed globally. Open up your command line or terminal:

npm install -g truffle

3. Create a new directory for the project and enter it.

mkdir leaseGenerator
cd leaseGenerator

4. Initialize a barebones truffle project.

truffle init

5. Create a package.json and package-lock.json

npm init -y
npm install

6. Install ganache-cli, a local test blockchain that mimics Ethereum.

npm install -g ganache-cli

7. Ensure you have a text editor installed (I prefer Visual Studio Code), then open up the project in the editor.

// if using VS code:
code .

8. The file/folder structure will appear as follows:

contracts/
migrations/
test/
package-lock.json
package.json
truffle-config.js

9. Now, install ethereum-bridge. This software allows us to use Provable on our local development blockchain, without deploying our contracts to an actual network.

npm install ethereum-bridge

10. Inside of package.json, create an scripts entry with the following:

"scripts": {
"bridge": "./node_modules/.bin/ethereum-bridge -a 9 -H 127.0.0.1 -p 8546 --dev",
"chain": "ganache-cli -p 8546 --defaultBalanceEther 10000 --db ./ganache/"
},

bridge is responsible for starting ethereum-bridge. the -a flag specifies which account to use on our local blockchain for deploying the Provable connector smart contract, which permits our isolated local project to communicate with the rest of the web. Then, we pass in the host and port that the blockchain is running on. --dev mode simply speeds up internal bridge tasks, making development faster.

chain spins up a local instance of ganache-cli on the specified host and port. --defaultBalanceEther equips each account with a starting balance of 10000 Ether, which will eventually be useful. --db sets a storage location to persist the data of ganache, making it possible to re-instantiate the same instance next time we run the startup command.

Truffle Configuration

Now we must edit the truffle configuration file to suit our purpose. Since we’re only developing locally, the config is simple. Open up truffle-config.js , delete the contents and write the following:

module.exports = {
networks: {
development: {
host: "127.0.0.1",     
port: 8546,            
network_id: "*",       
}
},
compilers: {
solc: {
version: "0.5.17",    
}
}
}

Here, we specify which networks Truffle will use to deploy smart contracts. development is a special keyword that tells Truffle it is the default network. Fill in the appropriate host and port. Provide * for network_id. solc is a solidity compiler that Truffle uses under the hood. Ensure the version  solc is set to 0.5.17. At the time of writing, the latest version of solidity is 0.6.4. We’re using the latest version of the 0.5 series because Provable hasn’t yet provided 0.6 compatibility.

Libraries

This project relies on two libraries, SafeMath & provableAPI. Create 2 new .sol files in /contracts & copy the source for each library.

Lease Generator Contract

Now that we’ve got our foundation built and configuration settings, it’s time to begin writing our contracts. Navigate into /contracts and create LeaseGenerator.sol

(cd contracts && touch LeaseGenerator.sol)

The parenthesis brings us back to our working directory which is a nice little hack. Open up LeaseGenerator.sol

Line 1 is the start of every new dApp project. Ethereum developers across the globe should cherish this moment as a symbol of new beginnings. Do not downplay the significance of declaring the solidity version at the top of the file!

Lines 2–3 imports our libraries.

Line 5 opens a new contract which inherits from usingProvable which contains all the logic for making data queries.

Line 7 declares our use of SafeMath with uint variable types. This enable us to tack on .add or .sub etc. to any uint type for simple, safe arithmetic.

Line 9 declares an address payable type variable which holds the landlord’s Ethereum address. payable is to ensure funds can be sent to the address.

Line 10 declares another address payable for the tenant in question. This variable is updated whenever the given tenant is performing an action such as paying a lease deposit.

Line 12 stores our ETH to USD conversion rate. This variable will be set every time we retrieve a new rate from the oracle.

Line 13 is used to set the amount sent by a tenant in wei for a deposit or monthly lease payment. It is then used to calculate the equivalent USD amount for updating the contract’s state.

Line 14 keeps track of the total wei amount received by the contract from monthly payments. When the landlord choses to withdraw funds from the contract, this is the amount that’s sent.

Lines 16–22 expresses an enum type, which is essentially a custom type that can be any of the values listed within the curly braces. Line 24 declares workingState as a variable with the type State we declared above. For every action that is performed such as paying a deposit, the workingState is updated to reflect it. For example, if a tenant is paying a lease deposit, we update workingState to payingLeaseDeposit. The purpose is to provide context for the oracle. After the oracle receives some requested data, it fires a callback function __callback() that allows us to customize how we handle the incoming data. This will be entirely dependent on the value of workingState. If we’re payingLeaseDeposit, the callback will direct instruction to pay the lease deposit.

Lines 26–36 declares the Lease object with all of its properties. A new instance of Lease is created when the landlord decides to create a new lease. Properties:

1. numberOfMonths : duration of the lease in months
2. monthsPaid : total months paid by the tenant
3. monthlyAmountUsd : total monthly amount to be paid in USD
4. leaseDepositUsd : value of the lease deposit to be paid
5. leasePaymentWindowSeconds : amount of time the tenant has to make a monthly payment in seconds, goes into effect after the tenant has paid the lease deposit
6. leasePaymentWindowEnd : unix timestamp deadline for the tenant to make a lease payment
7. depositPaymentWindowEnd : unix timestamp deadline for the tenant to make a lease deposit
8. leaseDepositPaid : tells whether or not the lease deposit has been paid
9. leaseFullyPaid : tells whether or not the entire lease has been fully paid

Line 38 stored mappings of IDs. A unique ID is returned every time a new oracle query is created. This ID is added to the mapping to be used by __callback() to ensure each query is processed only once.

Line 39 maps tenants to leases. When the landlord creates a lease, the given tenant address is stored here with a reference to the new lease instance so it may be retrieved when performing lease actions.

Lines 41–44 declare a modifier. We’ll attach this to functions we want to be called only by the landlord. It wouldn’t make sense if a random person could just empty the contract of its funds right?

Lines 46–85 declare events for all the actions that can be performed. Each event is fired after the corresponding action is complete. Such events are useful in a dApp for frontend code to respond and update the UI in response.

In the interest of creating shorter code snippets, the rest of the code will continue to be displayed as a new GitHub gist starting at line 1. Continue on as if the following gist comes after the previous one. This will come up a few more times throughout the tutorial.

Lines 1–5 set the constructor, which is the function that is executed when the contract is deployed. It is marked payable so that it can receive Ether. We will want to send some Ether upon initialization to pay for Provable queries.

Line 2 sets the landlord’s address to msg.sender, which is the address that deploys the contract.

Line 3 sets a custom gas price for Provable’s __callback() function. We set a high amount to ensure we can cover the cost of operations within __callback().

Line 4 is the Provable address resolver. When we launch ethereum-bridge, it deploys a contract that facilitates the communication of our contract to the outside world of the internet. The address of this contract is exposed so we can include it here in our contract. Now Provable can use this contract to perform its operations. Be prepared to modify the address to the output of eventually running npm run bridge. For now, leave it as is.

Lines 7–11 is the function responsible for telling Provable to fetch the USD rate of ETH.

Line 8 provable_getPrice("URL") asks Provable to calculate and return the amount of gas required to retrieve an external resource from a web URL. We use require to ensure that we have enough balance in the contract to cover that cost.

Line 9 provable_query tells Provable to execute the query with the given endpoint, which in our case is the ETH/USD price ticker from Coinbase. This returns a new query ID, which we add to validIds on line 10.

Lines 13–28 is our implementation of Provable’s __callback(). The first parameter is the ID produced from the query. The second is the actual result of the query as a string. The keyword memory is used to store the string in memory for the duration of the function, instead of writing it to storage, which is more gas cost effective.

Line 14 checks that the ID has been used by the query. If it hasn’t yet been used, an error will be thrown and execution will fail.

Line 15 ensures that the caller of __callback() is the address of the deployed provable contract as an added security feature. No random actor should be able to execute __callback().

Line 16 invalidates the ID so it cannot be used again by another query.

Line 17 parses the string result, transforms it into a uint type and assigns it to our global variable ETHUSD.

Lines 19–27 are a series of conditional statements relying on the value of workingState, which is set by the given function that called fetchUsdRate(). We haven’t yet implemented any of these. As you can see, the relevant function is called based on the state.

Line 1–33 is the function responsible for creating new leases. It takes in six parameters:

1. numberOfMonths : duration of the lease in months
2. monthlyAmountUsd : total monthly amount to be paid in USD
3. leaseDepositUsd : value of the lease deposit to be paid
4. leasePaymentWindowSeconds : amount of time the tenant has to make a monthly payment in seconds, goes into effect after the tenant has paid the lease deposit
5. depositPaymentWindowSeconds : amount of time the tenant has to make the deposit payment in seconds. Goes into effect immediately after the lease is created.
6. tenantAddr : the tenant’s Ethereum address, provided to the landlord via external communication

It’s public so it can be executed from an external Ethereum account and onlyLandlord so that only the landlord’s address can call it.

We use uint8, uint16, uint32 & uint64 because of a concept called ‘struct packing’. Once we create the lease instance from the Lease struct, using smaller uint types marginally decreases the gas costs.

Line 10 sets the actual unix timestamp to the value of now, which is the current value of the timestamp, plus the depositPaymentWindowSeconds. We use uint64() to ‘type cast’ the addition result (notice the .add), converting it to uint64 from uint256, because Safemath only works with uint256.

Lines 12–33 is creating a new Lease instance, initializing it with some data and assigning it to the value of the key tenantAddr in the tenantLease mapping storage. Refer to the explanation of the Lease object earlier on to re-enforce understanding.

Lines 24–33 emit the event leaseCreated with some information.

Line 35 to 44 is responsible for paying a lease deposit. It should be called by a tenant.

Line 36 grabs the Lease from storage, given the tenant’s address, which is msg.sender.

Line 37 ensures the lease deposit has not already been paid.

Line 38ensures the payment deadline for a lease deposit has not yet passed.

Line 40 sets the global variable tenantAddress to the tenant’s address.

Line 41 sets the global variable tenantPayment to the amount of ETH in wei the tenant has sent, msg.value.

Line 42 sets the workingState to payingLeaseDeposit. This is so that the callback fired after a result is given from calling fetchUsdRate() (line45) understands to call _payLeaseDeposit() to continue the flow.

Lines 46–64 represent the internal function responsible for continuing the process of paying a lease deposit, after the USD/ETH rate has been received. It is internal so that only __callback() can execute it after receiving the rate.

Line 47 resets the state to idle.

Line 48 retrieves the Lease based on tenantAddress set by calling payLeaseDeposit().

Line 49 takes the tenantPayment set by calling payLeaseDeposit(), multiplies it by the ETHUSD rate and finally divides it by 1e18. This division is to convert the wei value of the multiplication to a representation in Ether. This results in the amount sent by the tenant in USD.

Lines 51–54 ensure the amount sent in USD matches the lease’s leaseDepositUsd. We account for quick fluctuations in ETH price by adding an offset of $5. For example, if leaseDepositUsd is $500, the acceptable range of values for amountSendUsd is $495–$505.

Line 56 marks the lease’s deposit payment as paid.

Line 57 resets the deposit’s payment window end, because it has already been paid.

Line 58 sets the payment deadline for the actual monthly lease payment, which takes effect straight away. Again, notice the uint64 typecast so that we can take the addition value and store it in the Lease object properly.

Lines 60–63 emits the leaseDepositPaid event.

Line 1 is our public function to pay a lease.

Lines 2–5 should be self explanatory. We grab the Lease object as usual and perform some checks.

Lines 7–10 are identical to payLeaseDeposit(), except we set the state to payingLease.

Lines 13–43 declare the internal function for paying a lease.

Lines 14–16 are identical to payLeaseDeposit()

Lines 18–20 ensures that the amount sent in USD is at least the required monthly amount with a negative $5 offset. If lease.monthlyAmountUsd is $500, the minimum amountSentUsd must be $495. There isn’t an upper limit because the amount sent determines the amount of months paid. The tenant may pay multiple months at once.

Line 22 lease.monthsPaid and lease.monthlyAmountUsd are both cast to uint256 to ensure proper types for a safe .add. Here, we take the existing amount of months paid and the the amount of months currently being paid, which is obtained by dividing the monthly lease cost by 10 + the amount in USD. This 10$ grace is put in place to ensure the proper amount of months is recorded in case the value of amountSentUsd is slightly under lease.monthlyAmountUsd.

Line 23 casts the resulting value of the above operation to a uint8 so it can be stored in lease.monthsPaid.

Line 24 sets the global variable leaseBalanceWei to the original value of leaseBalanceWei plus the value of tenantPayment which is also expressed in wei. In the case the landlord wished to withdraw the lease payment, this value is the amount that will be sent.

Line 26–35 states that if the amount of months paid as a result of the current payment in progress has reached the amount of months agreed to in the lease, declare the lease as fully paid and emit an leaseFullyPaid event. The tenant has paid the lease in full. Otherwise:

Lines 35–42 simply resets the deadline for the next lease payment and emits an leasePaid event.

Line 1–10 declare our public function for collecting (repossessing) a lease deposit. It accepts the tenant’s address in question as to the only parameter and is marked onlyLandlord to ensure nobody else can steal lease deposits.

Line 2–5 is a repeat of previous logic.

Lines 7–9 are a repeat of previous logic.

Line 12–23 defines our internal function for completing the process of collecting a lease deposit.

Lines 13–14 are a repeat of previous logic.

Line 15 declares leaseDeposit and assigns it to lease.leaseDepositUsd. The purpose of this is to capture the value before we reset it to 0 on the next line (16). Then, on line 17, we transfer the captured leaseDeposit divided by the ETHUSD rate, multiplied by 1e18 to convert from wei, to the landlord. This pattern of capturing a value before setting the original storage location to 0 is to prevent a well-known smart contract vulnerability known as re-entrancy. There are plenty of great articles on security worth exploring!

Lines 19–22 emit the leaseDepositCollected event.

Line 1 instantiates our public function for reclaiming a lease deposit. This is to be called by a tenant, after their lease has been fully paid.

Lines 2–4 are repeated logic.

Lines 6–8 are repeated logic.

Line 11 begins the internal function to continue the process of reclaiming a lease deposit. The remainder of this function is repeated logic from _collectLeaseDeposit.

Lines 1–11 is the function that is called by the landlord to withdraw lease payments.

Line 2 ensures that the balance is greater than 0.

Lines 3–5 display the same withdrawal pattern we saw with _collectLeaseDeposit & _reclaimLeaseDeposit. The contract transfers the balance to the landlord.

Lines 7–10 emit the fundsWithdrawn event.

Lines 13–35 represent a utility function used to retrieve all the properties on an active Lease, given a tenant’s address.

Lines 13–22 tell the function which types it must return.

Line 23 brings up the Lease in question and allocates it to memory. It’s unnecessary to prepend it with storage because we’re not interested in modifying it, only reading from it.

Lines 24–34 tells the function to return all the properties on the Lease object.

Lines 37–39 are a utility function to retrieve the current ETHUSD rate.

Lines 41–43 declare another utility function to show the contract’s current balance. In order to get the balance, we must convert this (the contract instance) into an address type to then read the balance off of it.

Finally, line 45 is the fallback function. If the contract randomly receives Ether, perhaps from the landlord to refill for Provable queries, it receives it gracefully and adds it to its balance.

WHEW!!! What a journey. Here’s the complete Gist:

Now that we’ve completed the smart contract, let’s see it working in action. Head over to my Github repo and clone it to get access to the complete project with tests. In the last section of this tutorial, we will walk through how to run the tests and watch them pass.

The first step is to run npm run chain in the terminal. You’ll notice we’ve set the starting balance of each account to 10000 ETH. This is because the tests iterate through many scenarios of addresses spending lots of ETH. Once ganache-cli is running, run npm run bridge. When the bridge finishes booting up, it will display a message:

Please add this line to your contract constructor:OAR = OraclizeAddrResolverI(0x6f485C8BF6fc43eA212E93BBF8ce046C7f1cb475);

Copy the address displayed and paste it into the constructor of LeaseGenerator.sol.

In the terminal, run truffle test. The tests may fail if the price of ETH is rapidly fluctuating and the rate mismatch ends up producing an offset of more than $5 for the amountSentUsd. Just run them again. Also, while the tests are running, head into the terminal window running the bridge. Here, we can see all the requests being sent out and coming back live as the tests are running. Fascinating right? The ganache window also rapidly displays transactions as they are being processed.

I encourage all readers to look inside /tests/leaseGenerator.test.js to see how the tests are implemented. It shows how we deal with the huge amount of async activity involved in the contract interaction.

This application is far from perfect. It has many shortcomings that fail to address certain real-life circumstances. It is, however, an interesting take as to how a landlord/tenant relationship may play out in the future if this type of technology becomes mainstream.

Feel free to comment on clarification, improvements, considerations, etc.

Attribution

This article is contributed by Etienne Dusseault.

Also Read:

If you want to learn more about the Crypto ecosystem, sign up for the weekly newsletter.