logo
down
shadow

Issues with ave function in R: error "cannot allocate vector of size 419 kb."


Issues with ave function in R: error "cannot allocate vector of size 419 kb."

By : aldrin perez
Date : November 19 2020, 03:01 PM
wish help you to fix your issue For data in long format, I am trying to generate a sequence of 1:length of event to count length (time) of each event within ID, to look like this: , Using sequence as FUN may not work as expected. For e.g.
code :
sequence(c(6, 5))
#[1] 1 2 3 4 5 6 1 2 3 4 5

seq_along(c(6, 5))
#[1] 1 2
df1$Time <- with(df1, ave(Event, Event, ID, FUN = seq_along))
library(data.table)
setDT(df1)[, Time := seq_len(.N), .(ID, Event)]


Share : facebook icon twitter icon
Memory Allocation "Error: cannot allocate vector of size 75.1 Mb"

Memory Allocation "Error: cannot allocate vector of size 75.1 Mb"


By : kevmax
Date : March 29 2020, 07:55 AM
will help you R has gotten to the point where the OS cannot allocate it another 75.1Mb chunk of RAM. That is the size of memory chunk required to do the next sub-operation. It is not a statement about the amount of contiguous RAM required to complete the entire process. By this point, all your available RAM is exhausted but you need more memory to continue and the OS is unable to make more RAM available to R.
Potential solutions to this are manifold. The obvious one is get hold of a 64-bit machine with more RAM. I forget the details but IIRC on 32-bit Windows, any single process can only use a limited amount of RAM (2GB?) and regardless Windows will retain a chunk of memory for itself, so the RAM available to R will be somewhat less than the 3.4Gb you have. On 64-bit Windows R will be able to use more RAM and the maximum amount of RAM you can fit/install will be increased.
R lm function Error: cannot allocate vector of size 8.4 Mb

R lm function Error: cannot allocate vector of size 8.4 Mb


By : trying_To_Code
Date : March 29 2020, 07:55 AM
Does that help I have a huge data frame. I grouped all my data based on two columns The problem that when I use lm function with ddply I get an error Error: cannot allocate vector of size 8.4 Mb . However, when I use it for another functions as mean it works perfectly. Could you suggest me something that fix this problem, perhaps another function instead of ddply? I have used the maximum limit by the way , If you only need the coefficients, you can try this:
code :
library(data.table)
setDT(df)
dafr <- df[, as.list(lm.fit(cbind(1, b), a)$coef), by=list(c, d)]
setnames(dafr, c("c", "d", "intercept", "slope"))
#   c d    intercept slope
#1: 1 5 1.869449e-13   0.5
#2: 2 6 5.176935e-13   0.5
#3: 3 7 5.000000e+02   0.5
#4: 4 8 5.000000e+02   0.5
Left_join: Error: cannot allocate vector of size "small" Mb

Left_join: Error: cannot allocate vector of size "small" Mb


By : Khushboo Singhal Tha
Date : March 29 2020, 07:55 AM
this one helps. Since posting this question I have done some research. I first thought the errors had to do with the number(size) of objects in my workspace, which was not the case.
The most important answer to my own question (please feel free to elaborate on this), is that the size of the vector which cannot be allocated does not necessarily say a lot about what the operation does to memory.
code :
Error: cannot allocate vector of size 140.4 Mb
library(data.table)
df1 <- merge(df1, df2, by= "matchcode", all.x = TRUE, allow.cartesian=TRUE)
R (64-bit on Windows): How to get around "cannot allocate vector of size 557.6 Mb" error in code

R (64-bit on Windows): How to get around "cannot allocate vector of size 557.6 Mb" error in code


By : user3647806
Date : March 29 2020, 07:55 AM
With these it helps My first solution here was that to remove those rows from file1 and file2 where the USERID is blank.
This is not an ideal solution (considering that merge is probably more DT-friendly than the join functions in dplyr) as per comment made by @r2evans.
code :
library(tidyverse, data.table)

con1  <- file("file1.csv", open = "r")
con2  <- file("file2.csv", open = "r")

file1 <- select(read.csv(con1, sep = "|", fill = F, colClasses = "character"),
                PROFILE.ID, USERID)

file1 <- filter(file1, USERID != "")
setDT(file1)

file2 <- select(read.csv(con2, sep = "|", fill = F, colClasses = "character"),
                PROFILE.ID, USERID)

file2 <- filter(file2, USERID != "")
setDT(file2)

full_join(file1, file2, by = "USERID") %>%
  filter(is.na(PROFILE.ID.x) | is.na(PROFILE.ID.y) |
                PROFILE.ID.x != PROFILE.ID.y)

close(con1)
close(con2)
R : Any other solution to "cannot allocate vector size n mb" in R?

R : Any other solution to "cannot allocate vector size n mb" in R?


By : TheBidetShop
Date : March 29 2020, 07:55 AM
fixed the issue. Will look into that further The problem is that R tries to allocate 90mb of continuous space. Unfortunately, after many operations, it is possible that the memory is too fragmented.
If possible, try to optimize your code to use small chunks of data at a time.
Related Posts Related Posts :
  • Statistical mode of a categorical variable in R (using mlv)
  • Using for-loop to mutate a data.frame in r
  • Make plot with regression line for mixed model
  • Shortcut to select matces cases in R studio
  • vectoriced norm/matrix multiplication
  • Negative log10 transformation in R
  • Plot data with duplicate points
  • Visualizing crosstab tables with a plot in R - changing colours
  • How to manually modify automated numbers and labels in plot
  • How can I follow any redirections of a url in R?
  • Add jitter to box plot using markers in plotly
  • Adding an extra item to the legend
  • ggplot fills in data in the wrong order
  • Convert list to data frame
  • R: filtering by list(s) of strings and returning all results that start with the content of the lists
  • R:How to attach parts of a data frame with different headers and/or an overflowing piece of the dat frame
  • How to use 'par' for manipulating plot margins?
  • Can dplyr::case_when return mix of NAs and non-NAs?
  • Text preprocessing and topic modelling using text2vec package
  • Uploading multiple files in Shiny, process the files, rbind the results and return a download
  • R levelplot: color green-white-red (white on 0) according to one variable, but show the values of another variable
  • Why [i] doesn't point to the starting point in a vector
  • In R after generating a mvrnorm distribution, Y, what does Y[,1] do?
  • expand a data frame to have as many rows as range of two columns in original row
  • Getting started with R and CFA
  • Re order x-axis in ggplot so time goes from 12AM to 11PM in R
  • R - Automatically stack every nth column of a data frame and save them as new objects
  • How to format dplyr output in R into doubles (or other workable format)?
  • Dataframe to matrix conversion using tapply turns zeros to NAs
  • Smallest multiple of 1:20 - How can I make it quicker?
  • How to specify the size of a graph in ggplot2 independent of axis labels
  • How can I find the number of a vector's elements in another vector?
  • ROC curve from train/test set in caret R package
  • Random Forest for a mixture of categorical,numeric and "unwanted" variables which include missing values
  • extract certain data from multiple excel files with R
  • Matrix with counts of wins and losses between methods in R
  • Grouping string variables from a dataframe by best string match to make subsets
  • Reorder does not work after adding second geom_points
  • cover POS data formate to the one can apply Arules (Apriori)
  • Matching values between data frames based on overlapping dates
  • Grouped bar chart turns into stacked bar chart ggplot
  • R: How to fill in NA Values within a Column based on grouping?
  • Two action buttons, but only the first one, that is written in the server file, works?
  • Barchart grouped by variable both count up to 100 percent
  • Converting time in R to 24 hours
  • R - Web scrapping and downloading multiple zip files and save the files without overwriting
  • Find month and year inside string
  • Append multiple csv files into one file using R
  • Use `purrr::map` with k-means
  • R - 'data' is not an exported object from 'namespace:my_package'
  • Sum vector with number by dinamic intervals without looping
  • Shiny system call with continuous updates
  • Unable to un-nest some fields using google bigquery (standard)
  • How to perform row mean in matrix by 3 number of columns in sequence using R
  • Non absolute counts histogram for imbalanced groups
  • Plot_ly color not corresponding to z
  • Numeric calculations using dplyr piping commands
  • Separate a column with uneven/unequal strings and with no delimiters
  • Using lapply and which to subset dataframe by both characteristic and fuction
  • Removing factor levels from variable X based on values in Y
  • shadow
    Privacy Policy - Terms - Contact Us © voile276.org