Soil Moisture Data Analysis

languages: ennl

Data format

The data from the prototypes soil moisture sensors is saved in the ' Extra'  field of the Meet Je Stad database.

The values are seperated by a comma and are in the order: Cnc*, Tnc, Cref*, Tref, C1*, T1*, C2*, T2*, C3*, T3, C4*, T4

- The C values represent a rough value for the capacity of the soil
- The T values can be calculated into a temperature by T = Tx / 4 -20

The number in subscripts denotes at which electrode / depth the value is measured. The asterix indicates that the values are raw values and need to be converted to true capacity values.

- nc = not connected (capacity of the measurement circuit, no temperature sensor connected)
- ref = reference capacitor (39.000 femtofarad +/- 1%) and temperature sensor (NTC) on the PCB
- 1 = 10cm depth
- 2 = 40 cm depth
- 3 = 80 cm depth
- 4 = 120 cm depth

To obtain the capacity of the soil-electrodes (in femtoFarad), we need to substract from the raw values the raw value of the measurement circuit and multiply the result with the k-factor of the circuit. The k-factor can be calculated from the reference capacitor as such:

k = 39.000 / (Cref*-Cnc*)

The capacity of an electrode can then be calculated by:

Cx = (Cx* - Cnc*) * k

This is a first rough calculation, in which we for now, neglect the influence of temperature on the measured capacity values.


The aim of the pilot with soil moisture prototypes is to test the sensors outdoors and compare the data with data from professional sensors that are also placed on all the pilot locations. Prototypes need to be investigated for reliability of the data and how they can be best calibrated. 

Coated versus uncoated sensors (26-11-2020)

To (potentialy) solve issues with unexpected data patterns caused by moisture, we experimented with coating the PCB and the sensor electrodes with a silicone coating. (nodes 517 & 757). Based on the first data this coating seems to reduce the variability of capacity measurements of the cirucit and reference capacitor, and the k-value calculated from these measurements. Based on these first findings it was decided to revise sensors that were placed perviously. Appart from the above mentioned sensors, sensors connected to nodes 519, 661, 754 and 761 were taken appart, coated and re-assembled.

Analysis of the first data (8-11-2020) 

At this moment the data from the professional sensors is not yet available to perform an extensive analysis. Therefore we first looked at:
- rough calibration-measurement
- If there are unexpected patterns in the raw capacity measurements that may indicate sensor errors

Calibration measurements

Each sensor that has been build and placed at one of the pilot locations has been roughly calibrated by measuring the capacity in air and subsequently in water. The figure below depicts which factors contribute to the finally measured capacity. Here we distinguish between sensors in which loose wires run from the PCB to the electrodes and sensors in which a thicker flat cable is used. The circuit on the PCB (including the multiplexer), on average, has a capacity of 80pF and 70pF for the two types of sensors, respectively. Of this, 47pF is from the capacitor that is placed after the Schmitt trigger to ensure a sufficiently low frequency. The other electric components (Schmitt trigger itself, resistance, multiplexer and copper traces) hence contribute 23-33pF.

The capacity of the electrodes (10 to 120 cm) when measured in air, ranges from 8 to 28pF in the sensors with loose wires and from 8 to 39 in sensors with flat cable.De capaciteit van de electroden in lucht loopt van 8-28 pF in de sensoren met losse draden en van 8-39 in sensoren met flat cable. The thicker wires and proximity of individual wires in the flat cable, thus enhances the 'base'  capacity, particularly of the electrodes at 80 and 120cm depth.

The moment the sensors are placed, at each depth a soil sample is taken to determine the % soil moisture content. The added capacity by the dry soil is subsequently calculated from the first measurement taken after placing the sensor on location and the soil moisture content. On average this value was 11 pF.

The added capacity by water is 14.5 pF, when the electrode is completely surrounded by water. In reality the maximum volume of water that can reside in the soil is strongly dependent on the type of soil and can be up to 50% of the volume.

Hence the measurement range (50% of 14.5 )= 7pF is thus very small compared to the 'base' capacity of the electrode pairs. The capacity added by the soil is also remarkably high, as the relative elctrical permittivity of sand (dominant soil type in Amersfoort) is much smaller than water.
This rough calibration does not take into account the temperature-dependency of the different factors contributing to the final capacity measurement: circuit, reference-capacitor, electrodes, soil and water). Its well possible that the calculated capacity of the soil in fact is causes by the difference between the temperature at the moment the sensors were calibrated in water/air and the temperature of the soil the moment the sensors were placed. 

Next step: Analyse the temperature dependence of the different factors: circuit, reference-capacitor, electrodes, soil and water)


Example data

The graph below shows first data of the sensor connected to Met Je Stad node 517. In the left graph the raw capacity values are given of the circuit (green), the reference capacitor (orange) and the electrode at 10 cm depth (brown). The data shows that the raw capacity values of all three channels is fluctuating during the day, which indicates the temperature-dependence of the measurement. On 22-10 the batteries of the Meet je Stad node are depleted and the sensor provides some abnormal values. The temperature measurement at 10cm shows an expected pattern: it follows air temperature, but averages out the peaks.
Example data.png

Unexpected pattern / sensor error #1 

With some sensors (and specific channels) the raw capacity value fluctuates more strongly during the day compared to others. In the graph below the raw values for the circuit and the reference-capacitor are shown for three sensors. The k-value for the sensor connected to node 517 is very stable until just before the node runs out of batteries. The relationship between raw capacity values is lineair and negative.
Error 1_temp_dependance change.png
For the sensor connected to node #661 the same pattern occurs in the first month after placement, but after the node is restarted the temperature-dependence of the 'circuit'  channel changes to non-lineair and positive. This does not happen for the 'reference-capacitor'  channel. The capacity value of the 'circuit' channel is measured over a multiplexer channel that is not connected to an electrode. Because the 'reference-capacitor' channel is stable, the problem likely is caused by the multiplexer and not by the elctronics before the multiplexer.
The sensor connected to node #753 is an example in which this behavior is already visible directly after placement on location and is also specific for the 'circuit'  channel and not for the channel measuring the reference-capacitor or the electrode at 10cm depth.  The figure below shows the connection of the channels on the multiplexer. The ' circuit'  (NC) channel is located  distance form other channels and any problem with moisture may be local enough to influence only this channel and not others.
Error_1_multiplex locations.png

Unexpected pattern / sensor error #2

With some sensors we can see that the measured raw value makes a sudden jump, after which the preceding trend is maintained, but at a higher absolute level. In the graph below it can be seen that for the sensor connected to node #661, this occurs for the electrode at 80cm depth, but not for the electrodes at 40 and 120 cm.

Unexpected pattern / sensor error #3

Two sensors out of the 10 sensors that were placed on pilot locations, show very variable measurement results shortly after placement. The nodes also malfunction quickly after, indicating a potential role for leakage currents.