Journal of Mechanical Engineering and Automation
p-ISSN: 2163-2405 e-ISSN: 2163-2413
2018; 8(1): 7-31
doi:10.5923/j.jmea.20180801.02

Timothy Sands, Clinton Armani
Defense Advanced Research Projects Agency (DARPA), Arlington, VA, USA
Correspondence to: Timothy Sands, Defense Advanced Research Projects Agency (DARPA), Arlington, VA, USA.
| Email: | ![]() |
Copyright © 2018 The Author(s). Published by Scientific & Academic Publishing.
This work is licensed under the Creative Commons Attribution International License (CC BY).
http://creativecommons.org/licenses/by/4.0/

System identification algorithms use data to obtain mathematical models of systems that fits the data, permitting the model to be used to predict and design controls for system behavior beyond the scope of the data. Thus, accurate modeling and characterization of the system equations are very important features of any mission. These system equations are principally populated with variables from physics (e.g. material properties). Simple control algorithms begin by using the governing physics expressed in mathematical models for control, but usually more advanced techniques are required to mitigate noise, mismodeled system parameters, of unknown/un-modeled effects, in addition to disturbances. Characterization of the physical system a priori is an important step, and this research article will describe research into identifying system equations for several complex structures. Under the auspices of the Digital Manufacturing Analysis, Correlation and Estimation (DMACE) (pronounced “DEE-MACE”) Challenge, the Defense Advanced Research Projects Agency (DARPA) digitally manufactured several complex structures and then conduct a series of structural load tests upon them to determine material properties. Data from the manufacture and load tests was then posted on the worldwide web. Participants were challenged to develop a correlation model that accurately correlates digital manufacturing (DM) machine inputs to output structural test data. Participant models were then evaluated by their ability to predict the test results of the final DM structures. The model that most accurately predicted the final test results won the Challenge. Many disparate technical approaches were investigated by researchers from all over the world, and this paper introduces readers to several of those interesting technical approaches. The authors have permission to publish these government owned submissions, but every efforts is made to credit the researchers themselves, and furthermore each submission is presented in its original form to the maximum extent permitted by the journal’s peer reviewers and editors.
Keywords: Additive digital manufacturing, DARPA, Defense Advanced Research Projects Agency, DMACE Challenge, Digital Manufacturing Analysis, Correlation, and Estimation Challenge, DARPA Challenge
Cite this paper: Timothy Sands, Clinton Armani, Analysis, Correlation, and Estimation for Control of Material Properties, Journal of Mechanical Engineering and Automation, Vol. 8 No. 1, 2018, pp. 7-31. doi: 10.5923/j.jmea.20180801.02.
![]() | Figure 1. Participants in the DARPA DMACE Challenge |
![]() | Figure 2. Vacuum arc process for converting titanium into ingot [Seong] |
![]() | Figure 3. Converting titanium ingot into a part [Seong] |
![]() | Figure 4. Emerging technologies for titanium production [Seong] |
![]() | Figure 5. Titanium mill products producer price index trend [Seong] |
![]() | Figure 6. One-piece titanium mesh spheres directly from powder |
![]() | Figure 7. Load-test of one-piece titanium mesh spheres |
![]() | Figure 8. Nomenclature for digital manufacture of polymer cubes |
![]() | Figure 9. “Dogbone” samples used to evaluate DM polymer properties |
![]() | Figure 10. Sample CubeSat structures |
![]() | Figure 11. Final contest CubeSat structure being crushed |
Predicted strength for the sphere is
Math model for cubes, for the final question estimation:
Math model for cubes, for the 2nd practice question estimation, a area, t tip size, r raster angle, b build angle:p = g(t,r,b,a) = a*(59.430218+38.952698*t + 7.5636129*cos(0.372657+b-0.052924275*r) / (41.201572*t*t - 0.60272503))
![]() | Figure 12. Calculations of Sphere Model |
![]() | Figure 13. Calculations of Cube Model |
![]() | Figure 14. Sphere data extrapolation |
![]() | Figure 15. Sphere data extrapolation |
![]() | Figure 16. Sphere data extrapolation |

![]() | Figure 17. Unknown Cube configuration |
![]() | Figure 18. Known cube configurations |
![]() | Figure 19. Final test specimen |
![]() | (1) |
![]() | (2) |
|
|
![]() | Figure 20. Maximum force sustained versus energy/area |
The prediction for this set of parameters is
Sphere SummaryWhile this statistical method provides a means to predict the strength of this unique spherical structure, this exercise does not actually examine the resulting properties of the material formed. A more thorough test would have generated bar samples for material strength tests, as done in the other part of the challenge. Very little is known about the underlying structure of the material. It would have been instructive to have samples crusted at 45°.[1] 3.5.2 Prototype Method 2 - ThermoplasticThe second challenge tested a different manufacturing technique which involved creating various sample geometries, and cube-structures in a FORTUS 400mc thermoplastic 3-d printer with ABS-M30 thermoplastic. The manufacturing process involves a variable width nozzle that extrudes the thermoplastic material in layers. The final geometry to be tested was one roughly the size of a cube- satellite. Cube satellites are purpose built for a multitude of missions but they share the same external dimensions. Having an ability to build and predict performance of various structural designs would be highly desirable. Compression and tensile tests were performed on the thermoplastic samples to demonstrate the material properties of the various manufacturing orientations. Each set of samples differed by extrusion diameter, raster angle (the direction at which the nozzle moved while extruding the thermoplastic in the oven), and by build angle (the direction between the longitudinal axis of the part and the oven floor). Half of the samples were heat cured for 12 hours prior to test. Three cube structures were then produced with various open and closed wall designs, similar to many weight saving techniques used in aerospace structures. The first two were sample designs and the third was the challenge test subject. 2.1 The Model This method of manufacturing seems to have much higher repeatability as compared to the Arcam A2 machine. The test structure is a five sided box with solid sides. This structure can withstand 64890 ± 779.8867 N before failure. The contest asks us to predict a structure that has cutouts. The solid wall box can be estimated as being comprised of many adjacent columns. When the cutouts are made, the number of effective columns is reduced.In the test cutout box, the effective column count is reduced by 40% when four columns of 10 mm cutouts are made per side in the 100 mm per side box, the resulting strength should be reduced by 40%. This thinking leads to a prediction for this test structure of a maximum force of 38934 ± 467.93 N which is well within the agreement of the actual value which is 40616.91 ± 885.19 N. Note that the estimation is off by only -1.9.The Challenge ProblemThe challenge problem provides for a more complicated problem that the simple cutout challenge problem. The structure now has no bottom, has substantial cutouts of the four sides (with cross members) and has an inner spiral which serves to give a non uniform support to the walls. This structure is substantially different from the other two structures because it does not have any bottom. As such, it can be roughly modeled as four corner columns held in a configuration by the cross members. The box is loosely prevented from caving in by the spiral structure, although it is not simple to perform this calculation without advanced simulation software. The estimation therefore will seek to pull the entire strength of the structure from the four corner beams. Each corner beam is approximately 10mm on each side and 5mm thick. This Following the adjacent column model described previously, this would result in a total of 80mm of 5mm thick columns for the wall of the structure. The center of the spiral provides an additional 5mm by 5mm column, for a total of 85mm of effective 5mm column. The test box had effectively 400mm of 5mm column. Therefore, neglecting the absence of a bottom, this final challenge box should capable of support a force of 85mm/400mm X 64890.00N =
This estimation does not account for the fact that the structure is more likely to fail after rotating about Z, whereas the original test box did not express a rotational preference as a failure mode.Thermoplastic SummaryThis statistical method relied on more consistent sample sets; however each sample set only contained only two data points. Additional repetitions of each sample set would have improved the robustness of the model. More complex cube-sat geometries would require different lab tests to be conducted to test the shear-interface between multiple layers of thermoplastic.![]() | (3) |
![]() | (4) |
|
|

![]() | (5) |
![]() | (6) |

![]() | (7) |

![]() | Figure 21. Data fit to straight line |
(which I consider the best estimate with the available data). The previous explanations could certainly justify increasing or decreasing the result a little bit, but these errors are minor compared to the last step of the procedure outlined.While the standard deviation (SD) at low energy levels tended to be in the 4% range, as the data moved to higher energy levels the SD fell to around 1.5%. The regression line would have been even better. The limited 2.5 ma 0 degree data was less than 0.4%. However, when the 60 degree data was presented, the SD rose to 5%. I would have thought this was manufacturing out of control, except the 0 degree data came from the same batch and it was highly controlled. As a consequence, this large standard deviation is the largest source of error (by far) in the final result. I would look for a different method of generating this final step, but the data at 60 degrees is very limited and this appears to be the best procedure available.While I do not know what is causing this large SD in the 60 degree crushing tests, I would expect this same problem to exist in the final spheres as well which will give a lower sample Mahalanobis distance.![]() | Figure 22. The corners |
![]() | Figure 23. The corners |
![]() | Figure 24. The X-portion |
![]() | Figure 25. The X-portion |
![]() | Figure 26. The spiral |
![]() | Figure 27. The final result |

![]() | Figure 28. Cube geometries |
![]() | Figure 29. Sphere crushing behavior |

After data on the spheres crushed at 60˚ became available, we again tested our model. It gave an error of about 12% so we decided to go back and try to find a dependence on angle. Repeating the analysis, however, did not return any new significant variables. The dependence on Current^2 increased slightly so we included it in our final model:
Using this approximation the predicted value for a Sphere constructed at 2.5mA, 110mm/sec and crushed at an angle of 60° is 
![]() | Figure 30. Cube stress-strain |



![]() | Figure 31. Materials properties for ABS Tip 16 in tension and compression |
![]() | Figure 32. Geometry for the CUBE Challenge |
![]() | Figure 33. Mesh used for the ABAQUS simulation with shell-elements |
![]() | Figure 34. Load-deflection curve for the cube challenge problem. The inset shows contours of the plastic strain at the maximum load |
with
the test angle in radians,
the beam velocity in mm/s, and
the beam current, in mA. Four different polynomials were explored (of degrees 1 to 4), resulting in 50 fitting parameters.
On inception, the swarm objects considered every data point supplied (160+), but an identical result was obtained using just the batch-average values for each dataset (20) on the early 'practice' problems. This close correlation in predictive results indicated that the optimizer was indeed converging on useful coefficients and the extra compute-overhead of factoring all points in the clouds was unnecessary particularly in that we were predicting batch-average values. The average error was minimized over all batch data points. The average error taken as the Euclidean length of an n-dimensional vector composed of all the individual errors. Individual errors were determined as the difference in measured load relative to the corresponding predicted load. Optimizations were considered thorough and exhausted when populations in excess of 100,000 particles operating with generational life spans of 10,000 calculation cycles failed to find any predictive 'improvements' within a complete generation. Particle integration time step (value governing particle motion in normalized parameter space) was interacted with over the course of optimizations, being varied from 1.0 down to ~0.00001 as progress stagnated. Three different sets of calculations were executed, as described below. Set1: All 20 datasets were considered with the constraint that all coefficients must be positive. Although the error was generally higher than for the other simulations, higher order polynomials (quadratic and cubic) performed best, resulting in more realistic shapes in the variables space. Additionally, all predicted loads were positive. The prediction was viable when considered relative to the nearby values. SET1:Coefficients were permitted to take on values in the range [0,+10e6]LINEAR:Average error: 5.07175e3 Predicted load: 2.0266e4 (low)Coefficients:
QUADRATIC:Average error: 4.89191e+003Predicted load: 2.3030e+004 (reasonable)Coefficients:
CUBIC:Average error: 4.86730e+003Predicted load: 2.3454e+004 (reasonable)Coefficients:
QUARTIC:Average error: 4.57712e+008Predicted load: 1.4619e+007 (NOT REASONABLE!)Coefficients:
Set2: Only 6 datasets were considered with the constraint that all coefficients must be positive. The solution is similar to that of set 1, but with linear and quadratic polynomials emerging.Nearest Neighbors used (6 points):
LINEAR:Average error: 4.53880e+003Predicted load: 2.3750e+004 (reasonable)Coefficients:
QUADRATIC:Average error: 4.53880e+003Predicted load: 2.3750e+004 (reasonable)Coefficients:
CUBIC:Average error: 1.63838e+008Predicted load: 1.1703e+007 (NOT REASONABLE!)Coefficients:
QUARTIC:Average error: 8.45203e+008Predicted load: 1.7148e+008 (NOT REASONABLE!)Coefficients:
Set3: All 20 datasets were considered with no constraint on the sign of the coefficients. This resulted in the smallest average error of all sets (with cubic and quartic polynomials), but also allowed for predictions of negative load values; prediction through fitting was not viable. Average errors, predicted loads, and equation coefficients for each equation are reported below:Coefficients were permitted to take on values in the range [-10E6,+10E6]LINEAR:Average error: 1.82252e+003Predicted load: 2.3997e+004 (reasonable)Coefficients:
QUADRATIC:Average error: 3.05536e+002Predicted load: 2.9967e+004 (high)Coefficients:
CUBIC:Average error: 5.44815e+001Predicted load: -3.1866e+005 (BAD!)Coefficients:
QUARTIC:Average error: 2.42912e+001Predicted load: 5.8078e+004 (high)Coefficients:
Based on the above, our answer for the SPHERE CHALLENGE is
|
![]() | Figure 35. Challenge participants’ predictions (true value in red) |