i using setdt() add additional columns data.table
setdt(mydata)[, paste0('f2_e',2:30) := lapply(.sd, function(x) log(value/x)), .sdcols = 32:60][] is not being added when run script:
library(data.table) library(zoo) date = seq(as.date("2016-01-01"),as.date("2016-05-10"),"day") value =seq(1,131,1) mydata = data.frame (date, value) mydata setdt(mydata)[, paste0('f1',2:30) := lapply(2:30, function(x) rollmeanr(value, x, fill = rep(na,x-1)) ),][] setdt(mydata)[, paste0('f2',2:30) := lapply(2:30, function(x) rollapply(value,x,fun="median",align="right",fill=na))][] setdt(mydata)[, paste0('f1_e',2:30) := lapply(.sd, function(x) log(value/x) ), .sdcols = 3:31][] setdt(mydata)[, paste0('f2_e',2:30) := lapply(.sd, function(x) log(value/x)), .sdcols = 32:60][] rbind(colnames(mydata)) rbind(colnames(mydata)) [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13] [,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25] [,26] [,27] [1,] "date" "value" "f12" "f13" "f14" "f15" "f16" "f17" "f18" "f19" "f110" "f111" "f112" "f113" "f114" "f115" "f116" "f117" "f118" "f119" "f120" "f121" "f122" "f123" "f124" "f125" "f126" [,28] [,29] [,30] [,31] [,32] [,33] [,34] [,35] [,36] [,37] [,38] [,39] [,40] [,41] [,42] [,43] [,44] [,45] [,46] [,47] [,48] [,49] [,50] [,51] [,52] [,53] [,54] [1,] "f127" "f128" "f129" "f130" "f22" "f23" "f24" "f25" "f26" "f27" "f28" "f29" "f210" "f211" "f212" "f213" "f214" "f215" "f216" "f217" "f218" "f219" "f220" "f221" "f222" "f223" "f224" [,55] [,56] [,57] [,58] [,59] [,60] [,61] [,62] [,63] [,64] [,65] [,66] [,67] [,68] [,69] [,70] [,71] [,72] [,73] [,74] [,75] [,76] [,77] [1,] "f225" "f226" "f227" "f228" "f229" "f230" "f1_e2" "f1_e3" "f1_e4" "f1_e5" "f1_e6" "f1_e7" "f1_e8" "f1_e9" "f1_e10" "f1_e11" "f1_e12" "f1_e13" "f1_e14" "f1_e15" "f1_e16" "f1_e17" "f1_e18" [,78] [,79] [,80] [,81] [,82] [,83] [,84] [,85] [,86] [,87] [,88] [,89] [1,] "f1_e19" "f1_e20" "f1_e21" "f1_e22" "f1_e23" "f1_e24" "f1_e25" "f1_e26" "f1_e27" "f1_e28" "f1_e29" "f1_e30" you can see there no f2_e2, f2_e3,etc... columns.
why columns not added?
short answer:
use setdt(mydata) once, , separately. assignment statements.
additionally, if you're going add lot of columns use function alloc.col() over-allocate more slots up-front until next release (v1.9.8). i.e.,
setdt(mydata) truelength(mydata) # [1] 100 alloc.col(mydata, 1000l) truelength(mydata) # [1] 1000 in current development version, v1.9.7, we've increased over-allocation 1024, default. should happen extremely rarely.
a quick , detailed explanation:
this happens because data.table over-allocates column pointers during creation, , default over-allocation length 100 columns. can check truelength(). see ?truelength.
require(data.table) mydata = data.frame (x=1, y=2) setdt(mydata) ## convert data.table reference length(mydata) ## equals columns assigned # [1] 2 truelength(mydata) ## total number of column slots allocated # [1] 100 let's add 30 more columns way did.
setdt(mydata)[, paste0("z", 1:30) := 1l] length(mydata) ## [1] 32 truelength(mydata) ## [1] 100 and 30.
setdt(mydata)[, paste0("z", 31:60) := 1l] length(mydata) ## [1] 62 truelength(mydata) ## [1] 100 and 30.
setdt(mydata)[, paste0("z", 61:90) := 1l] length(mydata) ## [1] 92 truelength(mydata) ## [1] 100 now, next time this, we've add 30 more columns, have 8 more slots free. need create object more over-allocated slots, assign columns in mydata new object, , assign mydata. , handled internally , automatically user doesn't have keep track. next time do:
setdt(mydata)[, paste0("z", 91:120) := 1l] the function [.data.table realises needs over-allocate again, , so, , new columns added new object. issue assigning result new object mydata in parent frame of [.data.table. , done through assign() statement, accepts variable name character input, , setdt(mydata) isn't. re-assignment step fails , therefore over-allocation couldn't reflected original object. if you'd done mydata[, paste0(..) := ...] input object mydata name, , can used assign over-allocated result original object, , that's why suggestion @thelatemail work.
if advanced, upgrade devel version, , this'll go away, , unlikely happen (unless you'd want have more 1024 columns in data.table).
i've filed #1731 remind come , see if there other ways around case.
